NativeAOT Arm32 cross compile fails with linker errors

[trampster]

Describe the bug

Building hello world for arm32 from desktop linux (Ubuntu) fails with clang linker errors

To Reproduce

1. Create a hello world dotnet console project for nativeaot dotnet new console --aot
2. Install cross compiler toolchain

apt-get update && apt-get install -y clang llvm gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf

3. Build for arm32

dotnet publish -r linux-arm -c Release

-> fails with clang linker errors

Exceptions (if any)

Restore complete (0.2s)
  WearableDotnet failed with 2 error(s) (0.2s) → bin/Release/net9.0/linux-arm/WearableDotnet.dll
    clang : error : linker command failed with exit code 1 (use -v to see invocation)
    /tmp/DOTNET_CLI_HOME/.nuget/packages/microsoft.dotnet.ilcompiler/9.0.1/build/Microsoft.NETCore.Native.targets(376,5): error MSB3073: The command ""clang" "obj/Release/net9.0/linux-arm/native/WearableDotnet.o" -o "bin/Release/net9.0/linux-arm/native/WearableDotnet" -Wl,--version-script=obj/Release/net9.0/linux-arm/native/WearableDotnet.exports -Wl,--export-dynamic -gz=zlib -fuse-ld=bfd /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/sdk/libbootstrapper.o /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/sdk/libRuntime.WorkstationGC.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/sdk/libeventpipe-disabled.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/sdk/libstandalonegc-disabled.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/sdk/libstdc++compat.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/framework/libSystem.Native.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/framework/libSystem.IO.Compression.Native.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/framework/libSystem.Net.Security.Native.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/framework/libSystem.Security.Cryptography.Native.OpenSsl.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.1/sdk/libz.a --target=armv7-linux-gnueabihf -g -Wl,-rpath,'$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -ldl -lrt -lm -pie -Wl,-pie -Wl,-z,relro -Wl,-z,now -Wl,--eh-frame-hdr -Wl,--discard-all -Wl,--gc-sections" exited with code 1.

Build failed with 2 error(s) in 0.6s

Further technical details

.NET SDK: Version: 9.0.102 Commit: cb83cd4 Workload version: 9.0.100-manifests.88387b28 MSBuild version: 17.12.18+ed8c6aec5

Runtime Environment:
OS Name: ubuntu
OS Version: 22.04
OS Platform: Linux
RID: linux-x64
Base Path: /home/hughesd/.dotnet/sdk/9.0.102/

.NET workloads installed:
[maui-android]
Installation Source: SDK 9.0.100
Manifest Version: 9.0.51/9.0.100
Manifest Path: /home/hughesd/.dotnet/sdk-manifests/9.0.100/microsoft.net.sdk.maui/9.0.51/WorkloadManifest.json
Install Type: FileBased

[maui-tizen]
Installation Source: SDK 9.0.100
Manifest Version: 9.0.51/9.0.100
Manifest Path: /home/hughesd/.dotnet/sdk-manifests/9.0.100/microsoft.net.sdk.maui/9.0.51/WorkloadManifest.json
Install Type: FileBased

Configured to use loose manifests when installing new manifests.

Host:
Version: 9.0.1
Architecture: x64
Commit: c8acea2262

.NET SDKs installed:
8.0.406 [/home/hughesd/.dotnet/sdk]
9.0.102 [/home/hughesd/.dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.13 [/home/hughesd/.dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.1 [/home/hughesd/.dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.13 [/home/hughesd/.dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.1 [/home/hughesd/.dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
DOTNET_ROOT [/home/hughesd/.dotnet]

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

- The IDE (VS / VS Code/ VS4Mac) you're running on, and its version

[trampster]

After some more research it seems I need a cross compiler docker image with a sysroot set and to specify the -p:CppCompilerAndLinker option and the -p:SysRoot option.

I've tried using images from https://github.com/dockcross/dockcross as follows:

# Use a .NET SDK image to build the application
FROM dockcross/linux-armv7a AS build

# Install dotnet sdk dependencies
RUN apt-get update && \
    apt-get install --no-install-recommends -y software-properties-common && \
    apt-get install --no-install-recommends -y wget apt-transport-https && \
    apt-get clean && \
     rm -rf /var/lib/apt/lists/*

# Install specific sdk version
RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh && \
    chmod +x ./dotnet-install.sh && \
    ./dotnet-install.sh --version 9.0.101 && \
    rm -rf /var/lib/apt/lists/*

ENV DOTNET_ROOT=/root/.dotnet
ENV PATH=$PATH:$DOTNET_ROOT:$DOTNET_ROOT/tools
ENV DOTNET_CLI_HOME=/tmp/DOTNET_CLI_HOME
ENV DOTNET_CLI_TELEMETRY_OPTOUT=true

WORKDIR /src
COPY *.csproj ./
RUN dotnet restore
COPY . .
RUN dotnet publish -r linux-arm -c Release -p:CppCompilerAndLinker=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/bin/arm-cortexa8_neon-linux-gnueabihf-g++ -p:SysRoot=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/arm-cortexa8_neon-linux-gnueabihf/sysroot

But this doesn't work will the following errors:

 => ERROR [8/8] RUN dotnet publish -r linux-arm -c Release -p:CppCompilerAndLinker=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/bin/arm-cortexa8_neon-linux-gnueabihf-g++ -p:SysRoot=/usr/xcc/arm-cortexa8_  5.8s
------                                                                                                                                                                                                          
 > [8/8] RUN dotnet publish -r linux-arm -c Release -p:CppCompilerAndLinker=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/bin/arm-cortexa8_neon-linux-gnueabihf-g++ -p:SysRoot=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/arm-cortexa8_neon-linux-gnueabihf/sysroot:                                                                                                                                                               
0.341   Determining projects to restore...                                                                                                                                                                      
3.374   Restored /src/WearableDotnet.csproj (in 2.89 sec).                                                                                                                                                      
4.678   WearableDotnet -> /src/bin/Release/net9.0/linux-arm/WearableDotnet.dll                                                                                                                                  
4.749   grep: /usr/xcc/arm-cortexa8_neon-linux-gnueabihf/arm-cortexa8_neon-linux-gnueabihf/sysroot/etc/os-release: No such file or directory
4.764   Generating native code
5.374 arm-cortexa8_neon-linux-gnueabihf-g++ : error : unrecognized command-line option '--target=armv7-linux-gnueabihf' [/src/WearableDotnet.csproj]
5.374 /tmp/DOTNET_CLI_HOME/.nuget/packages/microsoft.dotnet.ilcompiler/9.0.0/build/Microsoft.NETCore.Native.targets(376,5): error MSB3073: The command ""/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/bin/arm-cortexa8_neon-linux-gnueabihf-g++" "obj/Release/net9.0/linux-arm/native/WearableDotnet.o" -o "bin/Release/net9.0/linux-arm/native/WearableDotnet" -Wl,--version-script=obj/Release/net9.0/linux-arm/native/WearableDotnet.exports -Wl,--export-dynamic -gz=zlib -fuse-ld=bfd /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/sdk/libbootstrapper.o /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/sdk/libRuntime.WorkstationGC.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/sdk/libeventpipe-disabled.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/sdk/libstandalonegc-disabled.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/sdk/libstdc++compat.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.Native.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.IO.Compression.Native.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.Net.Security.Native.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/framework/libSystem.Security.Cryptography.Native.OpenSsl.a /tmp/DOTNET_CLI_HOME/.nuget/packages/runtime.linux-arm.microsoft.dotnet.ilcompiler/9.0.0/sdk/libz.a --sysroot="/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/arm-cortexa8_neon-linux-gnueabihf/sysroot" --target=armv7-linux-gnueabihf -g -Wl,-rpath,'$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -ldl -lrt -lm -pie -Wl,-pie -Wl,-z,relro -Wl,-z,now -Wl,--eh-frame-hdr -Wl,--discard-all -Wl,--gc-sections" exited with code 1. [/src/WearableDotnet.csproj]
------
Dockerfile:26
--------------------
  24 |     RUN dotnet restore
  25 |     COPY . .
  26 | >>> RUN dotnet publish -r linux-arm -c Release -p:CppCompilerAndLinker=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/bin/arm-cortexa8_neon-linux-gnueabihf-g++ -p:SysRoot=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/arm-cortexa8_neon-linux-gnueabihf/sysroot
  27 |     
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c dotnet publish -r linux-arm -c Release -p:CppCompilerAndLinker=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/bin/arm-cortexa8_neon-linux-gnueabihf-g++ -p:SysRoot=/usr/xcc/arm-cortexa8_neon-linux-gnueabihf/arm-cortexa8_neon-linux-gnueabihf/sysroot" did not complete successfully: exit code: 1

The first is because the sysroot doesn't have a os-release file (no idea how to fix this) or if i actually matters.

The second and probably most important is the following:
unrecognized command-line option '--target=armv7-linux-gnueabihf'

I think this may be because dockcross doesn't call it that but rather:

[root:/work] # arm-cortexa8_neon-linux-gnueabihf-g++ -dumpmachine
arm-cortexa8_neon-linux-gnueabihf

As far as I can tell there is no way for me to change the target that dotnet tries to use, and I can't see a way to convince dockcross to use armv7-linux-gnueabihf.

I have also tried the dockcross/armv7a but that uses armv7-unknown-linux-gnueabi which also doesn't match