Debugging a nuget package built by Azure Devops doesn't work

Let me start by saying if I build the package locally and publish it to a local drive, everything works as expected. If the same steps are done in Azure DevOps and the package is published to our private/company feed, debugging does not work (the code steps through the package code without going in). Also, this is .Net 4.8 application, not net core. Here's what I've done to enable debugging.

1. In the class library being packaged, installed Microsoft.SourceLink.GitHub packages to generate source link for GitHub repository (our code is there).
2. In VS, in Options->Debugging unchecked "Enable Just My Code".

I build a release configuration of the package either using VS build or msbuild (makes no differencce). Then I use Package Manager Console to build the package (i.e., nuget pack [path to project file] -properties configuration=Release). When I add this locally build package, debugging works just fine. My assumption is the above source link package points to the location in the repository. There's no pdb anywhere so, since everything works fine, I'm assuming it's not needed.

If the same steps are done on Azure Devops server, the package is published to the company's private feed and added to the consuming project from there, debugging does not work. You'd think with the project file being the same, debugging should work regardless where you built it.

I've tried to add "Index sources and publish symbols" step per these instructions but it makes no difference. I also tried making the below change to the project file setting DebuggingSymbols=true and DebugType=Full. This didn't make any difference either.

 <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugType>full</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release\</OutputPath>
    <DefineConstants>TRACE</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <Prefer32Bit>false</Prefer32Bit>
    <DebugSymbols>true</DebugSymbols>
  </PropertyGroup>

Please share your ideas!

The pipeline is below:

# Variable 'BuildVerbosity' was defined in the Variables tab
# Variable 'ProjectName' was defined in the Variables tab
trigger:
  branches:
    include:
    - refs/heads/main
resources:
  repositories:
  - repository: self
    type: git
    ref: refs/heads/main
jobs:
- job: Job_1
  displayName: Agent job 1
  pool:
    vmImage: windows-latest
  steps:
  - checkout: self
    clean: true
    fetchTags: false
  - task: Assembly-Info-NetFramework@3
    name: SetVersion
    displayName: Set Assembly Version
    inputs:
      FileNames: >-
        **\AssemblyInfo.cs

        **\AssemblyInfo.vb
      VersionNumber: '#.#.#.$(Build.Build开发者_运维知识库Id)'
      FileVersionNumber: '#.#.#.$(Build.BuildId)'
      FailOnWarning: true
  - task: Cache@2
    displayName: Cache NuGet Packages
    inputs:
      key: 926bec2e-d618-425e-b160-a368ecdcafc0
      path: packages
  - task: NuGetToolInstaller@1
    displayName: Get the latest NuGet.exe
  - task: NuGetCommand@2
    displayName: NuGet restore
    inputs:
      feedRestore: 697f4f9f-e201-40a9-9f9b-611dc64afec3
  - task: MSBuild@1
    displayName: Build solution **/*.sln
    inputs:
      msbuildArchitecture: x64
      configuration: $(BuildConfiguration)
  - task: CmdLine@2
    displayName: Display Directories - Delete this later
    enabled: False
    inputs:
      script: >
        echo "Here is the artifact directory: =====>>>   $(Build.ArtifactStagingDirectory)"

        echo "current directory"

        dir

        echo " "

        echo "*******************************************"

        echo "Build.SourcesDirectory\Projectname -> "  $(Build.SourcesDirectory)\$(ProjectName)\*

        dir $(Build.SourcesDirectory)\$(ProjectName)\*

        echo " "

        echo "*******************************************"

        echo "Build.SourcesDirectory\Projectname\bin\release\* -> "  $(Build.SourcesDirectory)\$(ProjectName)\bin\release\*

        dir $(Build.SourcesDirectory)\$(ProjectName)\bin\release\*

        echo " "

        echo "*******************************************"


        echo "Build.ArtifactStagingDirectory directory"

        dir $(Build.ArtifactStagingDirectory)
  - task: SimpleNuspecVersionSetter@1
    displayName: Set Nuspec Version to Beta
    inputs:
      RootDirectory: $(Build.SourcesDirectory)\$(ProjectName)
      NewVersion: $(SetVersion.AssemblyInfo.Version)-beta
  - task: NuGetCommand@2
    displayName: Create Package - Beta Version
    inputs:
      command: pack
      outputDir: $(Build.ArtifactStagingDirectory)/@BetaPackage
  - task: SimpleNuspecVersionSetter@1
    displayName: Set Nuspec Version to Release
    inputs:
      RootDirectory: $(Build.SourcesDirectory)\$(ProjectName)
      NewVersion: $(SetVersion.AssemblyInfo.Version)
  - task: NuGetCommand@2
    displayName: Create Package - Release Version
    inputs:
      command: pack
      outputDir: $(Build.ArtifactStagingDirectory)/@ReleasePackage
  - task: PublishSymbols@2
    displayName: Publish symbols path
    inputs:
      IndexSources: false
      SymbolServerType: TeamServices
  - task: PublishBuildArtifacts@1
    displayName: 'Publish Artifact: drop'
...