Build task dependencies vs target framework

[dazinator]

So I have noticed the following issue!

I have a very simple repro project, that consumes GitVersionTask
I have created multiple branches, in each branch I have amended the project to target a particular target framework/s. The name of the branch is named after the target frameworks that the project is targeting.
These branches all build on appveyor.

I have found that in some scenarios, the task does not run.

Target Framework	Runs ?
netstandard1.1	No
netstandard1.3	No
netstandard1.5	Yes
netcoreapp20;netstandard1.1	Yes

You can see all the relevant builds here: https://ci.appveyor.com/project/dazinator/gitversion-pr1269/history

So what is interesting is that the task doesn't seem to run until the consumer project is targeting atleast netstandard1.5 or a platform that is compatible with netstandard1.5 (i.e netcoreapp20).

I think this is down to the fact that the GitVersionTask nuget package has the following in its nuspec:

<dependencies>
      <group targetFramework=".NETStandard1.5">
         <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" exclude="Build,Analyzers" />
         <dependency id="GitTools.Core" version="$gittoolscoreversion$" exclude="Build,Analyzers" />
         <dependency id="YamlDotNet" version="$yamldotnetversion$" exclude="Build,Analyzers" />        
      </group>
      <group targetFramework=".NETFramework4.6.1">
         <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" exclude="Build,Analyzers" />      
      </group>
    </dependencies>

The sole reason this is there is because we rely on UtilPack to use this information to know how to restore / and resolve dependencies for us prior to executing the task.
However this is a problem. By listing dependencies in the nuspec we are effectively telling MsBuild to ingore the package if te consumer is targeting an incompatible framework. This is not what we want as the tool is a build tool, it's dependency chain is fixed based on the build environment, and not the target framework that a consumer is targeting.

[dazinator]

Perhaps we should include some additional nuspec inside the nuget package (or embedded inside the task assembly) explicitly for util pack to use to restore the tool at runtime?

This would mean the official nuspec for GitVersion task nuget package would have no dependencies. MsBuild would therefore not care about it or try to elimate it from the build based on the consumers target framework. However inside the package I include a UtilPack.nuspec - and then pass the path to this nuspec file to the TaskFactory if necessary, and util pack would use that to restore the task instead?

[dazinator]

I know that there is also a new DotNetCliTool reference to use for build tools instead of PackageReference, but I havent looked into whether this could potentially solve this issue.

[stazz]

I did some experimenting, with the gitversion-pr1269 stuff you gave me in other issue. It seems that when you change the target framework and do the build, the obj/repro.csproj.nuget.g.targets file always will contain the correct <Import Project="$(NuGetPackageRoot)gitversiontask\4.0.0-pullrequest1269-1530\build\GitVersionTask.targets" Condition="Exists('$(NuGetPackageRoot)gitversiontask\4.0.0-pullrequest1269-1530\build\GitVersionTask.targets')" /> element, even for netstandard1.1 target.

However, the element <Target Name="WriteVersionInfoToBuildLog" BeforeTargets="CoreCompile;GetAssemblyVersion;GenerateNuspec" Condition="$(WriteVersionInfoToBuildLog) == 'true'"> inside the GitVersionTask only gets triggered on the next build. I think this is one of the known issues with .target files that attach to currently running build: the attachment gets registered only starting from the following build which which wrote to the obj/repro.csproj.nuget.g.targets file. I can't find the link to the issue right now, but I remember reading about it.

By building twice, I was able to get the GitVersionTask to run even in netstandard1.1 build. I think the .nuspec file will play role in the actual build only if you put something in lib folder inside your NuGet package, which of course never happens for the MSBuild-task-supplying packages.

Can you get the GitVersionTask to run in all your builds if you run dotnet /t:Restore project.csproj and dotnet /t:Build project.csproj separately? You might need to delete the bin and obj folders before running Restore to ensure it will generate all correct stuff.

[stazz]

About DotNetCliTool - it seems like a good feature, and probably something that was intended exactly for this scenario (NuGet packages participating in the build, but not in runtime). However, it is currently not very well supported by e.g. GUI - meaning the consumer projects of your package will need to edit their .csproj file manually. Also, you would need to change your .nuspec file to have something like this:

    <packageTypes>
      <packageType name="DotnetCliTool" />
    </packageTypes>

I need to explore the DotNetCliToolReference later some more to see if it is mature and suitable enough for these kinds of purposes.

[dazinator]

Can you get the GitVersionTask to run in all your builds if you run dotnet /t:Restore project.csproj and dotnet /t:Build project.csproj separately? You might need to delete the bin and obj folders before running Restore to ensure it will generate all correct stuff.

Nope. All of the appveyor builds I have setup start completely clean, and do this:

build_script:
- cmd: dotnet restore ./src/repro.sln --disable-parallel
- cmd: dotnet build ./src/repro.sln --disable-parallel
- cmd: dotnet pack ./src/repro.sln --output ../../artifacts/

The end result is that, the task fires for some (seemingly dependent on the target framework being netstandard1.5 comaptible) but not others..

I could try adding in an additional dotnet build ?

[stazz]

Hmmmm.. I did dotnet command on .csproj file instead of .sln file, maybe that has an effect?

Edit: I tried out .sln as well, still running the targets in GitVersionTask for target frameworks netstandard1.1, netstandard1.3, and net40 (included old desktop framework just for testing). I ran the Restore and Build separately.

Oh wait! There is difference in our commands... I always use dotnet msbuild, while you seem to use dotnet restore, dotnet build. Maybe that's the difference?

Edit2: I tried dotnet restore and dotnet build instead of dotnet msbuild /t:Restore and dotnet msbuild /t:Build, respectively, and I still get the targets in GitVersionTask to run successfully (I get a bunch of [NuGet Minimal]: Restoring packages for C:\Users\me\AppData\Local\Temp\oxxm5v34.bsl\dummy... messages). What is the dotnet version on appveyor server?

[dazinator]

Hmm it works for me locally with no special effort
I wonder..
For me locally msbuild version is: Microsoft (R) Build Engine version 15.6.82.30579 for .NET Core
I will check appveyor

[stazz]

OK - also make sure it won' try to run different builds in parallel on same folder, although that probably should cause more bizzarre error messages.

Mine is the same:

> dotnet msbuild /version
Microsoft (R) Build Engine version 15.6.82.30579 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

15.6.82.30579

[dazinator]

AppVeyor msbuild version is: Microsoft (R) Build Engine version 15.5.180.51428 for .NET Core
I wonder if something changed between those two versions..
I would test that locally but I am not sure of the best way of installing previous versions of msbuild!

[stazz]

By creating global.json file in the current folder or any parent folder and putting

{
  "sdk": {
    "version": "2.1.4"
  }
}

your dotnet msbuild /version command should print 15.5.180.51428. However, everything still works for me even with that version. But it certainly does sound that something on the appveyor side is amiss, since it works locally for both of us. I guess you could always try to update the appveyor dotnet to newest, if possible, and see if it still happens... If you and me are both on Windows and AppVeyor is Linux, there might be something with that too (which is hopefully fixed in newest .NET Core).

Edit: The global.json file example of course assumes you have a 2.1.4 installation of SDK sitting around on your computer. :) Although maybe dotnet is smart enough these days to download it itself...

[dazinator]

Ok I have just managed to reproduce this.. and I am pretty sure I have figured out the problem in the process..

BTW AppVeyor has also seemingly applied the latest VS2017 update to their images since yesterday so the version of dotnet sdk is now handily the same as our local versions.

This is the problem:

On an AppVeyor environment, its a clean slate, there are no NuGet packages in the nuget package cache.

1. dotnet restore restores GitVersionTask for the target framework of the project (Netstandard1.1)
2. This downloads and installs GitVersionTask nuget package into the nuget packages directory - however as there are no dependencies listed in the nuspec under NetStandard1.1 for UtilPack, then the UtilPack package isn't downloaded and installed. This is the GitVersionTask nuspec:

          <group targetFramework=".NETStandard1.5">
         <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" exclude="Build,Analyzers" />
         <dependency id="GitTools.Core" version="$gittoolscoreversion$" exclude="Build,Analyzers" />
         <dependency id="YamlDotNet" version="$yamldotnetversion$" exclude="Build,Analyzers" />        
      </group>
      <group targetFramework=".NETFramework4.6.1">
         <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" exclude="Build,Analyzers" />      
      </group>

Therefore UtilPack nuget package is never downloaded and installed! So no wonder the targets don't work!

It worked for us because we have the package installed :-)
It also explains why when I added netcoreapp2.0 or a framework compatible with netstandard1.5 it started working again - because this causes the UtilPack dependency to be installed.

So it looks like I need to change the nuspec to add this:

<dependencies>
      <group>
        <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" />
      </group>
      <group targetFramework=".NETStandard1.5">
         <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" exclude="Build,Analyzers" />
         <dependency id="GitTools.Core" version="$gittoolscoreversion$" exclude="Build,Analyzers" />
         <dependency id="YamlDotNet" version="$yamldotnetversion$" exclude="Build,Analyzers" />        
      </group>
      <group targetFramework=".NETFramework4.6.1">
         <dependency id="UtilPack.NuGet.MSBuild" version="$utilpackversion$" exclude="Build,Analyzers" />      
      </group>
    </dependencies>

I am hoping this will fix this issue, and then I will close. One to be careful of :-)

[stazz]

Nice catch! 👍 I tested almost-this-kind-of-scenario by just removing GitVersionTask from my NuGet package repository, but I never realized to remove UtilPack.NuGet.MSBuild.

Let me know if the workaround is successful. I'll sit down and revisit this issue on better time tho to think of proper solution. :)

[dazinator]

Workaround successful! :-)

[stazz]

Workaround successful! :-)

Good to hear! I'll leave this issue open until there is some more closure about what could be the proper solution. :)

[dazinator]

According to the issue: GitTools/GitVersion#1454 - the introduction of these dependencies to our nuspec file, has meant that the nuget package is no longer deemed "compatible" with native projects! Just one to consider - I haven't fully investigated it but it seems plausible given our nuspec changes.