Cannot build on macOS

[patrikrazem]

I followed your developer guide and attempted to build the project on macOS by using:

./build.sh -- /p:SkipTests=true

which resulted in the following error:

<pathToRepo>/wcf/external/runtime/runtime.depproj(78,5): error MSB3073: The command "chmod +x <pathToRepo>/wcf/bin/testhost/netcoreapp-OSX-Debug-x64/dotnet" exited with code 1.

(I've replaced the real absolute path with <pathToRepo>).

This fails, because there is no such file in the targeted directory.

I've tried to run the build on Windows and it runs without any problems.

[Lxiamail]

@patrikrazem have you run "sync.sh"?

[patrikrazem]

I ran sync.sh manually, but the error message is still the same. There is no dotnet binary in the directory, so the chmod command fails. I'm attaching the whole output of the sync.sh script for reference:

> ./sync.sh
Installing dotnet cli...
Restoring BuildTools version 2.1.0-rc1-03131-06...
Initializing BuildTools...
Making all .sh files executable under Tools.
  Restoring packages for wcf/Tools/crossgen/crossgen.csproj...
wcf/Tools/crossgen/crossgen.csproj : warning NU1603: Microsoft.NETCore.DotNetHostPolicy 2.0.3 depends on Microsoft.NETCore.DotNetHostResolver (>= 2.0.3) but Microsoft.NETCore.DotNetHostResolver 2.0.3 was not found. An approximate best match of Microsoft.NETCore.DotNetHostResolver 2.0.4 was resolved.
wcf/Tools/crossgen/crossgen.csproj : warning NU1603: Microsoft.NETCore.DotNetHostPolicy 2.0.3 depends on runtime.osx-x64.Microsoft.NETCore.DotNetHostPolicy (>= 2.0.3) but runtime.osx-x64.Microsoft.NETCore.DotNetHostPolicy 2.0.3 was not found. An approximate best match of runtime.osx-x64.Microsoft.NETCore.DotNetHostPolicy 2.0.4 was resolved.
  Generating MSBuild file wcf/Tools/crossgen/obj/crossgen.csproj.nuget.g.props.
  Generating MSBuild file wcf/Tools/crossgen/obj/crossgen.csproj.nuget.g.targets.
  Restore completed in 2.32 sec for wcf/Tools/crossgen/crossgen.csproj.
Running crossgen on all assemblies in wcf/Tools.
wcf/Tools/Microsoft.DotNet.PlatformAbstractions.dll -> wcf/Tools/Microsoft.DotNet.PlatformAbstractions.ni.dll
wcf/Tools/System.Composition.AttributedModel.dll -> wcf/Tools/System.Composition.AttributedModel.ni.dll
wcf/Tools/project.dll -> wcf/Tools/project.ni.dll
wcf/Tools/System.Composition.Runtime.dll -> wcf/Tools/System.Composition.Runtime.ni.dll
wcf/Tools/Desktop.Analyzers.dll -> wcf/Tools/Desktop.Analyzers.ni.dll
wcf/Tools/System.Security.Cryptography.Hashing.Algorithms.Analyzers.dll -> wcf/Tools/System.Security.Cryptography.Hashing.Algorithms.Analyzers.ni.dll
wcf/Tools/GenFacades.Core.dll -> wcf/Tools/GenFacades.Core.ni.dll
wcf/Tools/NuGet.LibraryModel.dll -> wcf/Tools/NuGet.LibraryModel.ni.dll
wcf/Tools/NuGet.Packaging.Core.dll -> wcf/Tools/NuGet.Packaging.Core.ni.dll
wcf/Tools/GenFacades.exe -> wcf/Tools/GenFacades.ni.exe
wcf/Tools/GenAPI.exe -> wcf/Tools/GenAPI.ni.exe
wcf/Tools/System.Composition.Convention.dll -> wcf/Tools/System.Composition.Convention.ni.dll
wcf/Tools/csc.exe -> wcf/Tools/csc.ni.exe
wcf/Tools/NuGet.Versioning.dll -> wcf/Tools/NuGet.Versioning.ni.dll
wcf/Tools/System.Composition.Hosting.dll -> wcf/Tools/System.Composition.Hosting.ni.dll
wcf/Tools/run.exe -> wcf/Tools/run.ni.exe
wcf/Tools/System.Composition.TypedParts.dll -> wcf/Tools/System.Composition.TypedParts.ni.dll
wcf/Tools/vbc.exe -> wcf/Tools/vbc.ni.exe
wcf/Tools/vbc.dll -> wcf/Tools/vbc.ni.dll
wcf/Tools/csc.dll -> wcf/Tools/csc.ni.dll
wcf/Tools/NuGet.DependencyResolver.Core.dll -> wcf/Tools/NuGet.DependencyResolver.Core.ni.dll
wcf/Tools/Microsoft.Build.Tasks.CodeAnalysis.dll -> wcf/Tools/Microsoft.Build.Tasks.CodeAnalysis.ni.dll
wcf/Tools/NuGet.Configuration.dll -> wcf/Tools/NuGet.Configuration.ni.dll
wcf/Tools/ApiCompat.exe -> wcf/Tools/ApiCompat.ni.exe
wcf/Tools/NuGet.Common.dll -> wcf/Tools/NuGet.Common.ni.dll
wcf/Tools/System.Text.Encoding.CodePages.dll -> wcf/Tools/System.Text.Encoding.CodePages.ni.dll
wcf/Tools/System.Xml.XPath.XmlDocument.dll -> wcf/Tools/System.Xml.XPath.XmlDocument.ni.dll
wcf/Tools/Microsoft.DotNet.Build.CloudTestTasks.dll -> wcf/Tools/Microsoft.DotNet.Build.CloudTestTasks.ni.dll
wcf/Tools/NuGet.Frameworks.dll -> wcf/Tools/NuGet.Frameworks.ni.dll
wcf/Tools/BclRewriter.exe -> wcf/Tools/BclRewriter.ni.exe
wcf/Tools/NuGet.ProjectModel.dll -> wcf/Tools/NuGet.ProjectModel.ni.dll
wcf/Tools/Microsoft.DotNet.VersionTools.dll -> wcf/Tools/Microsoft.DotNet.VersionTools.ni.dll
wcf/Tools/Microsoft.Cci.Extensions.dll -> wcf/Tools/Microsoft.Cci.Extensions.ni.dll
wcf/Tools/MSBuild.dll -> wcf/Tools/MSBuild.ni.dll
wcf/Tools/Microsoft.DotNet.Build.Tasks.dll -> wcf/Tools/Microsoft.DotNet.Build.Tasks.ni.dll
wcf/Tools/NuGet.Packaging.dll -> wcf/Tools/NuGet.Packaging.ni.dll
wcf/Tools/Microsoft.Build.Utilities.Core.dll -> wcf/Tools/Microsoft.Build.Utilities.Core.ni.dll
wcf/Tools/Microsoft.DotNet.Build.Tasks.Packaging.dll -> wcf/Tools/Microsoft.DotNet.Build.Tasks.Packaging.ni.dll
wcf/Tools/NuGet.Commands.dll -> wcf/Tools/NuGet.Commands.ni.dll
wcf/Tools/Microsoft.Build.Tasks.Core.dll -> wcf/Tools/Microsoft.Build.Tasks.Core.ni.dll
wcf/Tools/NuGet.Protocol.dll -> wcf/Tools/NuGet.Protocol.ni.dll
wcf/Tools/Newtonsoft.Json.dll -> wcf/Tools/Newtonsoft.Json.ni.dll
wcf/Tools/Microsoft.Build.dll -> wcf/Tools/Microsoft.Build.ni.dll
wcf/Tools/Microsoft.Cci.dll -> wcf/Tools/Microsoft.Cci.ni.dll
wcf/Tools/Microsoft.CodeAnalysis.dll -> wcf/Tools/Microsoft.CodeAnalysis.ni.dll
wcf/Tools/Microsoft.CodeAnalysis.CSharp.dll -> wcf/Tools/Microsoft.CodeAnalysis.CSharp.ni.dll
wcf/Tools/Microsoft.CodeAnalysis.VisualBasic.dll -> wcf/Tools/Microsoft.CodeAnalysis.VisualBasic.ni.dll
Crossgen finished.
Done initializing tools.
Running: wcf/Tools/msbuild.sh /nologo /verbosity:minimal /clp:Summary /maxcpucount /l:BinClashLogger,Tools/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log  /flp:v=normal;LogFile=sync.log /t:Sync
  chmod: wcf/bin/testhost/netcoreapp-OSX-Debug-x64/dotnet: No such file or directory
wcf/external/runtime/runtime.depproj(72,5): error MSB3073: The command "chmod +x wcf/bin/testhost/netcoreapp-OSX-Debug-x64/dotnet" exited with code 1.
  wcf/packages/System.Reflection.DispatchProxy/4.5.0/lib/netcoreapp2.0/System.Reflection.DispatchProxy.dll (System.Reflection.DispatchProxy.4.5.0) -> wcf/bin/Unix.AnyCPU.Debug/corefx/netcoreapp/System.Reflection.DispatchProxy.dll
  wcf/packages/System.Security.Principal.Windows/4.5.0/lib/netstandard2.0/System.Security.Principal.Windows.dll (System.Security.Principal.Windows.4.5.0) -> wcf/bin/Unix.AnyCPU.Debug/corefx/netcoreapp/System.Security.Principal.Windows.dll

  Build FAILED.

wcf/external/runtime/runtime.depproj(72,5): error MSB3073: The command "chmod +x wcf/bin/testhost/netcoreapp-OSX-Debug-x64/dotnet" exited with code 1.
    0 Warning(s)
    1 Error(s)

[patrikrazem]

Btw, we've tested this on a few different macOS systems and the error is always the same.

[StephenBonikowsky]

Hi @patrikrazem the build process is susceptible to artifacts that get left behind from previous build attempts.

Please try running clean.sh -all first and then...
sync.sh -p
build.sh

[patrikrazem]

@StephenBonikowsky I followed the steps you provided, but the result is still the same. The chmod command fails, because there is no file to operate on.

[StephenBonikowsky]

@patrikrazem sync.sh calls run.sh which in turn calls init-tools.sh

You can run init-tools.sh directly, it should create a 'tools' directory into which it downloads a bunch of things needed for the build. One of these is 'dotnet.exe' which should be under '/tools/dotnetcli/' if it succeeds which it doesn't in your case.

The script attempts to download dtnet.exe using wget first and if that fails then it tries with curl. See lines 55 through 83

You should be able to try that manually and get an exact error, my guess is that it is something to do with wget or curl on your machine.

[patrikrazem]

@StephenBonikowsky Well, as far as I can tell init-tools.sh is contained inside the Tools directory, so I don't see how it could create it.

But I don't think that's the problem. I can see that dotnet has been downloaded and is present in /tools/dotnetcli/. I can run the binary /tools/dotnetcli/dotnet and it runs normally, displaying the usual usage help.

But the chmod command that fails is not trying to manipulate /tools/dotnetcli/dotnet.

Based on the output:

..... <lost of other stuff> .....
Crossgen finished.
Done initializing tools.
Running: wcf/Tools/msbuild.sh /nologo /verbosity:minimal /clp:Summary /maxcpucount /l:BinClashLogger,Tools/Microsoft.DotNet.Build.Tasks.dll;LogFile=binclash.log  /flp:v=normal;LogFile=sync.log /t:Sync
  chmod: wcf/bin/testhost/netcoreapp-OSX-Debug-x64/dotnet: No such file or directory
wcf/external/runtime/runtime.depproj(72,5): error MSB3073: The command "chmod +x wcf/bin/testhost/netcoreapp-OSX-Debug-x64/dotnet" exited with code 1.
  wcf/packages/System.Reflection.DispatchProxy/4.5.0/lib/netcoreapp2.0/System.Reflection.DispatchProxy.dll (System.Reflection.DispatchProxy.4.5.0) -> wcf/bin/Unix.AnyCPU.Debug/corefx/netcoreapp/System.Reflection.DispatchProxy.dll
  wcf/packages/System.Security.Principal.Windows/4.5.0/lib/netstandard2.0/System.Security.Principal.Windows.dll (System.Security.Principal.Windows.4.5.0) -> wcf/bin/Unix.AnyCPU.Debug/corefx/netcoreapp/System.Security.Principal.Windows.dll

you can see that Done initializing tools. is printed, which I think is this line. This suggests that init-tools.sh runs successfully.

I'm guessing that /tools/dotnetcli/dotnet should be copied into /bin/testhost/netcoreapp-OSX-Debug-x64/dotnet on build, but for some reason that doesn't happen? That's why the chmod command fails, since the file is not present there.

[wfurt]

I think that is problem with restoring packages. With /p:SkipTests=true runtime for tests will not be restored and the chmod than fails.