Roslyn analyzer execution incompatible with preview version difference

[mthalman]

dotnet/dotnet#4801 caused a regression for all source build legs. It's attempting to execute an analyzer targeting a new preview version (11.0 P3) using an older SDK version (11.0 P1).

Exec
    Assembly = /__w/1/s/.dotnet/sdk/11.0.100-preview.1.26104.118/Microsoft.Build.Tasks.Core.dll
    Parameters
        Command = "/__w/1/s/.dotnet/dotnet" "/__w/1/s/src/roslyn/artifacts/bin/CodeStyleConfigFileGenerator/Release/net11.0/CodeStyleConfigFileGenerator.dll" "CSharp" "/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CSharp.CodeStyle.Fixes/Release/netstandard2.0" "Microsoft.CodeAnalysis.CSharp.CodeStyle.targets" "/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CodeStyle/Release/netstandard2.0/Microsoft.CodeAnalysis.CodeStyle.dll;/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CSharp.CodeStyle/Release/netstandard2.0/Microsoft.CodeAnalysis.CSharp.CodeStyle.dll"
    CommandLineArguments = "/__w/1/s/.dotnet/dotnet" "/__w/1/s/src/roslyn/artifacts/bin/CodeStyleConfigFileGenerator/Release/net11.0/CodeStyleConfigFileGenerator.dll" "CSharp" "/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CSharp.CodeStyle.Fixes/Release/netstandard2.0" "Microsoft.CodeAnalysis.CSharp.CodeStyle.targets" "/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CodeStyle/Release/netstandard2.0/Microsoft.CodeAnalysis.CodeStyle.dll;/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CSharp.CodeStyle/Release/netstandard2.0/Microsoft.CodeAnalysis.CSharp.CodeStyle.dll"
    Errors
        /__w/1/s/src/roslyn/src/CodeStyle/CSharp/CodeFixes/Microsoft.CodeAnalysis.CSharp.CodeStyle.Fixes.csproj(34,5): error MSB3073: The command ""/__w/1/s/.dotnet/dotnet" "/__w/1/s/src/roslyn/artifacts/bin/CodeStyleConfigFileGenerator/Release/net11.0/CodeStyleConfigFileGenerator.dll" "CSharp" "/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CSharp.CodeStyle.Fixes/Release/netstandard2.0" "Microsoft.CodeAnalysis.CSharp.CodeStyle.targets" "/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CodeStyle/Release/netstandard2.0/Microsoft.CodeAnalysis.CodeStyle.dll;/__w/1/s/src/roslyn/artifacts/bin/Microsoft.CodeAnalysis.CSharp.CodeStyle/Release/netstandard2.0/Microsoft.CodeAnalysis.CSharp.CodeStyle.dll"" exited with code 150. [/__w/1/s/src/roslyn/src/CodeStyle/CSharp/CodeFixes/Microsoft.CodeAnalysis.CSharp.CodeStyle.Fixes.csproj]
    Messages
        You must install or update .NET to run this application.
        App: /__w/1/s/src/roslyn/artifacts/bin/CodeStyleConfigFileGenerator/Release/net11.0/CodeStyleConfigFileGenerator.dll
        Architecture: x64
        Framework: 'Microsoft.NETCore.App', version '11.0.0-preview.3.26124.119' (x64)
        .NET location: /__w/1/s/.dotnet/
        The following frameworks were found:
          11.0.0-preview.1.26104.118 at [/__w/1/s/.dotnet/shared/Microsoft.NETCore.App]
        Learn more:
        https://aka.ms/dotnet/app-launch-failed
        To install missing framework, download:
        https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=11.0.0-preview.3.26124.119&arch=x64&rid=linux-x64&os=centos.10

Example failed build (internal link)

[mthalman]

The regression was caused by dotnet/dotnet#4801 but the PR build validation succeeded there. Here's the Copilot analysis on why there's a difference between the PR build and the internal official build:

Root Cause: Runtime Version Mismatch

The failure is caused by a framework version mismatch between the compiled tool binaries and the installed .NET runtime.

Both builds share the same SDK and runtime (11.0.0-preview.1). The critical difference is the value of MicrosoftNETCoreAppRefPackageVersion set in PackageVersions.roslyn.Current.props:

	Failing Build	Successful Build
MicrosoftNETCoreAppRefPackageVersion	11.0.0-preview.3.26124.119	11.0.0-ci

How the failure occurs

1. PackageVersions.roslyn.Current.props in the failing build sets package versions to 11.0.0-preview.3.26124.119.
2. Tool projects (CodeStyleConfigFileGenerator, GenerateDocumentationAndConfigFilesForBrokenRuntime) compile targeting net11.0 with a framework dependency on Microsoft.NETCore.App 11.0.0-preview.3.26124.119.
3. When the build tries to execute these tools via Exec tasks, the .NET host looks for that runtime version.
4. Only 11.0.0-preview.1.26104.118 is installed on the build agent.
5. .NET does not roll forward across preview bands, so the tools fail with exit code 150 (framework not found).

Why the successful build works

In the successful build, PackageVersions.roslyn.Current.props sets MicrosoftNETCoreAppRefPackageVersion = 11.0.0-ci, which resolves to the locally installed preview.1 runtime. The tools compile against a compatible runtime and execute successfully.

[mthalman]

@ericstj - Just want to check with you on the priority of this. This is impacting all the source build legs in official builds. This should get resolved soon or the original change should be reverted.

[ericstj]

We should just undo a change like this. Sorry I hadn't realized that PR merged yet, I was waiting on merging it as I was validating in another build.

[ericstj]

It's attempting to execute an analyzer

It's a plain old application, not an analyzer. Analyzers are insulated by from this sort of thing by targeting netstandard2.0.

Only 11.0.0-preview.1.26104.118 is installed on the build agent.
.NET does not roll forward across preview bands, so the tools fail with exit code 150 (framework not found).

Roll-forward is not an option. They'd need to roll backward -- which isn't a thing. The motivation for dotnet/dotnet#4801 was to fix cases where repos were picking up the latest packages, but not the latest framework references. This causes issues because they'll be in torn state when we make dependent changes. If they pick up the latest packages, they need to pick up the latest framework references too. If they then turn around and run code during the build -- they need to run that code against the latest runtime as well.

We'll usually have tools like this target an LKG and not the "latest" framework -- to avoid getting in this torn state while not forcing the build to deploy the just built runtime for build tool usage.

Source build is forcing this repo to build against the latest framework for this.

I'll undo the change for now to unblock things. We can redo the change and run the internal build pipeline to validate that it works correctly before merging again.

[ericstj]

Here are a couple tools that do this in roslyn.
https://github.com/search?q=repo%3Adotnet%2Froslyn+%2FExec+.*ArtifactsBinDir%2F&type=code

I think the solution is to condition when we apply those properties. I don't think we need to actually stage a runtime for downstream execution.

[ericstj]

Reverted the break to unblock the build dotnet/dotnet#5116

Testing a fix - could be more of these types of tools in other repos, so might need to peel the onion.
https://dev.azure.com/dnceng/internal/_build/results?buildId=2913885&view=results