dotnet build and test commands fail in nondeterministic way when using the Microsoft.NET.Test.Sdk package

[mars-low]

Description

Steps to reproduce

I've prepared a minimal reproducible example with CI running on Github Actions at public repository https://github.com/mars-low/msbuild-flat-project-hierarchy and also attached binlog file directly. I've stumbled upon this issue for the first time around a year ago, when I was porting .NETFramework project to .NET5 and forgot about it until I run into it again on .NET6. It occurs in nondeterministic way, but I've managed to reproduce it on CI by running dotnet build a few times in a row.

Expected behavior

I'd like to build and execute test project residing next to referenced project using single dotnet test command.

Actual behavior

dotnet test and dotnet build commands fail with the following error in a nondeterministic way (on CI it failed only after seven successful builds):

Error: /home/runner/.nuget/packages/microsoft.net.test.sdk/17.2.0/build/netcoreapp2.1/Microsoft.NET.Test.Sdk.Program.cs(3,12): error CS0234: The type or namespace name 'VisualStudio' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) [/home/runner/work/msbuild-flat-project-hierarchy/msbuild-flat-project-hierarchy/ProjectTests.csproj]

I know about three potential workarounds for this issue:

1. Move test project and referenced project to separate directories. It is the cleanest solution, but requires modification to the project structure.
2. Add Microsoft.NET.Test.Sdk package also to referenced project. It seems not natural as all tests are contained only in a test project.
3. Do not use dotnet build and dotnet test commands directly. It is cumbersome as I expect to run all tests with single dotnet test command. Instead run three separate commands in the following order:

dotnet restore ProjectTests.csproj
dotnet msbuild ProjectTests.csproj 
dotnet test --no-build ProjectTests.csproj

Initially I was going to report in dotnet/sdk repository, because I found an issue describing the same problem dotnet/sdk#14147, but I think that it is especially connected with the Microsoft.NET.Test.Sdk package. I don't get build errors if I replace Microsoft.NET.Test.Sdk with other package in example mentioned at the beginning. Also changing the order of tags in project file doesn't seem to help.

Diagnostic logs

I provided binlog file obtained as an artifact from the CI using command dotnet build -bl ProjectTests.csproj
structured-log.zip

Environment

I've tested it on Github Actions runner: https://github.com/mars-low/msbuild-flat-project-hierarchy/runs/7436609249?check_suite_focus=true so I'm pasting a top header from the setup stage:
Current runner version: '2.294.0'
Operating System: Ubuntu 20.04.4 LTS
Virtual Environment
Environment: ubuntu-20.04
Version: 20220717.1
Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20220717.1/images/linux/Ubuntu2004-Readme.md
Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20220717.1
Virtual Environment Provisioner: 1.0.0.0-main-20220701-2

[nohwnd]

msbuild.zip
I can still repro even with 17.10 this just by repeating this commands above few times.

1..20 | % { dotnet restore ProjectTests.csproj; dotnet msbuild ProjectTests.csproj -bl ; dotnet test --no-build ProjectTests.csproj }

@MichalPavlik do you see something that we do obviously wrong? The test.sdk nuget package uses this target to include one of the files in the same folder.

https://github.com/microsoft/vstest/tree/main/src/package/Microsoft.NET.Test.Sdk/netcoreapp

But in some cases the type for the attribute is not found, the type is defined in our ObjectModel dll which comes from another nuget.

Binlog is in the zip.

[nohwnd]

I can repro with both dotnet msbuild and dotnet build, so it is most likely our fault.