[Traversal] Support Platforms as a First Class Concept

[benvillalobos]

See the first example of dotnet/msbuild#7760 (comment), TraversalPlusP2PToVcxproj.zip.

This includes a traversal project that simply references all project recursively, and an AnyCPU csproj that references a vcxproj with explicit SetPlatform metadata.
Expected: When the dirs.proj is built, the vcxproj builds once, as its only platform (x64).
Traversal projects should build all platforms of the projects they directly reference.
Actual: The vcxproj builds twice: once (successfully) because of the csproj with the explicit SetPlatform metadata, and once more (failure) because the dirs.proj didn't supply any Platform global property.

The ask: The Traversal SDK should support Platforms as a first class concept, and attempt to build all ProjectReference items based on what's found in Platforms.

Questions:

• What platform should be chosen for each referenced project? dirs.proj's typically don't have a "preferred platform" (correct me if I'm wrong), so I'm not sure how we would "just know" what to build a project as.

cc @AArnott

[AArnott]

What platform should be chosen for each referenced project? dirs.proj's typically don't have a "preferred platform" (correct me if I'm wrong), so I'm not sure how we would "just know" what to build a project as.

I believe a dirs.proj should invoke each referenced project, for all its supported platforms. Unlike any other P2P where the originator presumably already has a Platform set, which is used in negotiation, a dirs.proj is responsible to build "everything", so when no Platform/SetPlatform metadata is set on the P2P in a dirs.proj, it should just build every platform supported by that project.

[AArnott]

The Traversal SDK should support Platforms as a first class concept, and attempt to build all ProjectReference items based on what's found in Platforms.

Do you mean the dirs.proj itself will set the Platforms property, or that it will support reading this property from referenced projects in order to invoke them? I think setting Platforms on dirs.proj would be weird, because it has no output.

FWIW I think some of our dirs.proj in the VS repo have platform subsets defined in properties, but I think this is just a relic and that dirs.proj should build whatever projects they reference, all platforms by default. Any variation on this them like limiting platforms built to subsets could be optional additional properties or metadata set in the dirs.proj.

[MeikTranel]

@AArnott so basically like this?

dirs.proj

<Project Sdk="Microsoft.Build.Traversal">
  <ItemGroup>
    <ProjectReference Include="a/a.csproj" />
    <ProjectReference Include="b/b.csproj" />
  </ItemGroup>
</Project>

Basic Execution Diagram

Build -->dirs.proj
  --> dirs.proj (PopulatePlatformSpecificProjectReferences)
    ------(GetPlatforms)---> a.csproj
    ------(GetPlatforms)---> b.csproj
  --> dirs.proj (Build)
    ----------(Build)--------> a.csproj (rid:win-x86)
    ----------(Build)--------> a.csproj (rid:win-x64)
    ----------(Build)--------> b.csproj (rid:win-x86)

[AArnott]

Yes, although the global property set on a.csproj would be Platform=x86 rather than RuntimeIdentifier=win=x86.
If we want to talk about building all RIDs, that would be a separate issue. This is about building all platforms via the same platform enumeration mechanism as is used for the recent platform negotiation feature.

[MeikTranel]

recent platform negotiation feature

is there documentation or somewhere i can read up on?

To my knowledge Platform, RuntimeIdentifier & PlatformTarget are hanging in a weird nirvana right now where nobody really can define them and documentation work is being done to define a path forward. To my knowledge Platform is a concept that never existed outside of Windows platforms & formerly Xamarin target platforms. So there is no way to define for example ARM M1 within the Platform/PlatformTarget sphere - or linux based arm64, or is there?

[AArnott]

is there documentation or somewhere i can read up on?

Sure: https://github.com/dotnet/msbuild/blob/main/documentation/ProjectReference-Protocol.md#setplatform-negotiation

It does seem to me that RID is more descriptive and perhaps less problematic than Platform. Nevertheless Platform is a very well established concept in msbuild projects and what we need to work with first. RIDs can and I believe do tend to follow Platform's lead. For example a Debug|x86 setting for Config|Platform can end up setting the RID to win-x86. Now, if you also had linux-x86 as a rid, you'd have to create a new project platform for that (or more likely, a new debug_linux config) and set the RID based on that yourself.

But in VS at least, config|platform are the only dimensions we get to switch around on. Maybe one day RID will be an option or replace Platform, but until then, we need platform support.

[MeikTranel]

Got it - thought NearestTargetFramework was wild - little did i know.

Back to the feature request - when you list a project that references a library project that is not listed in the dirs.proj. Would you expect that Library to be also queried and matrix build for all platforms available or just the ones referenced in the dirs.proj?

[AArnott]

MSBuild projects cannot be influenced by external projects except via P2P. So when a project references a library project, the way its P2P behaves cannot vary based on the presence of a dirs.proj.
As defined in platform negotiation already, an ordinary project that references another project should build one platform in the referenced project, the one being selected as compatible and most specific. A dirs.proj which may or may not exist can have no influence on this.

A dirs.proj has no platform, and is generally used to build a whole set of projects just to make sure they build, which leads me to suggest that they should build everything (all platforms) in each project.

[jeffkl]

@benvillalobos Traversal projects are just a project that has ProjectReference items for build scheduling while the traversal project itself doesn't emit any output. How does a C# project-to-project reference work in MSBuild today with Platforms? Traversal projects should just tell MSBuild that a dependent project needs to be built and the .NET SDK should schedule the inner builds for each target framework and platform right?

[AArnott]

the .NET SDK should schedule the inner builds for each target framework and platform right?

For each target framework yes the SDK does this today. But platforms do not all get inner-builds for them.
For example, if I just build a lone project with dotnet build some.csproj, all target frameworks for the project will build, but only its default platform will build.
This is something I would like to see change over time. I'd like to see platforms participate in this inner-build as well so that I can build a project completely, including all platforms (or RIDs) with just one top-level call. But that feature request is independent of this request.

The intersection of the two is perhaps interesting though. Because if/when that feature request is ever delivered, dirs.proj might implicitly have the behavior of building all platforms for a referenced project by default. But as the feature request is unlikely to be fulfilled in the foreseeable future, and the requested behavior in this issue can be done more cheaply and with far less concern for backward compatibility, it makes sense IMO to do this.