arm* debbuilds can fail to get image with specified platform

[scpeters]

While building new Jetty packages with *gz-jetty* package aliases, I have noticed several failures with messages like

failed to get destination image "sha256:...": image with reference sha256:... was found but does not provide the specified platform (linux/arm64)

Sometimes it passes on a retry, sometimes it fails.

Example build results:

gz-math 9.2.0-2

• gz-math 9.2.0-2(ubuntu/noble::amd64) https://build.osrfoundation.org/job/gz-math9-debbuilder/384/
• gz-math 9.2.0-2(ubuntu/noble::armhf) https://build.osrfoundation.org/job/gz-math9-debbuilder/385/
• gz-math 9.2.0-2(ubuntu/noble::arm64) https://build.osrfoundation.org/job/gz-math9-debbuilder/386/

Step 51/54 : USER $USER
 ---> Running in 79911b94e642
 ---> Removed intermediate container 79911b94e642
 ---> bbaa9177c8f6
Step 52/54 : COPY --chown=$USER:$USER . /home/jenkins/workspace/gz-math9-debbuilder/.
failed to get destination image "sha256:bbaa9177c8f6c6248720248f957766f1bc6e7f9b418c7d55a3eb18e4a9e1d76c": image with reference sha256:bbaa9177c8f6c6248720248f957766f1bc6e7f9b418c7d55a3eb18e4a9e1d76c was found but does not provide the specified platform (linux/arm64)

• retry gz-math 9.2.0-2(ubuntu/noble::arm64) https://build.osrfoundation.org/job/gz-math9-debbuilder/387/

18:27:05 Step 51/54 : USER $USER
18:27:07  ---> Running in c67d0a9ff534
18:27:07  ---> Removed intermediate container c67d0a9ff534
18:27:07  ---> bad3971a3618
18:27:07 Step 52/54 : COPY --chown=$USER:$USER . /home/jenkins/workspace/gz-math9-debbuilder/.
18:27:07 failed to get destination image "sha256:bad3971a3618dbd98b8e0090661cb7a08a001c1842792f76ea3815e3548bb9e9": image with reference sha256:bad3971a3618dbd98b8e0090661cb7a08a001c1842792f76ea3815e3548bb9e9 was found but does not provide the specified platform (linux/arm64)

gz-tools 2.0.3-2

• gz-tools 2.0.3-2(ubuntu/resolute::amd64) https://build.osrfoundation.org/job/gz-tools2-debbuilder/3638/
• gz-tools 2.0.3-2(ubuntu/resolute::arm64) https://build.osrfoundation.org/job/gz-tools2-debbuilder/3639/
• gz-tools 2.0.3-2(ubuntu/resolute::armhf) https://build.osrfoundation.org/job/gz-tools2-debbuilder/3640/

Step 11/52 : RUN echo "OSRF repositories configured: stable stable"
 ---> Running in 88a197280f96
OSRF repositories configured: stable stable
 ---> Removed intermediate container 88a197280f96
 ---> 1706623067f5
Step 12/52 : ADD https://api.github.com/repos/gazebo-tooling/gzdev/git/refs/heads/master version.json

failed to get destination image "sha256:1706623067f55a571594e6ffb025f78d835a58f6ac240b10968b16eb6993a5ae": image with reference sha256:1706623067f55a571594e6ffb025f78d835a58f6ac240b10968b16eb6993a5ae was found but does not provide the specified platform (linux/arm)

• retry: * gz-tools 2.0.3-2(ubuntu/resolute::armhf) https://build.osrfoundation.org/job/gz-tools2-debbuilder/3641/

[j-rivero]

Thanks for the report. Investigating this:

Two arm architectures share one Jenkins agent, and since #1521 they also share one Docker build-cache chain. The second arch to build on a given agent adopts the first arch's layers.

It is happening since before b80b3bb (#1521), arm64 used FROM osrf/ubuntu_arm64:${DISTRO} while armhf used FROM ubuntu:${DISTRO}. Different roots, no collision. #1521 moved both to ubuntu:${DISTRO}, making the Dockerfiles identical.

It is flaky since some of the agents are ephemeral swarm nodes. Land on a node the other arm arch just used → fail. Land on a fresh node → pass.

[scpeters]

I am still seeing failures even with #1530 in place:

gz-common7 7.3.0-2(ubuntu/resolute::armhf)

https://build.osrfoundation.org/job/gz-common7-debbuilder/401/

Step 12/53 : RUN echo "OSRF repositories configured: stable stable"
 ---> Running in a6402ec91a05
OSRF repositories configured: stable stable
 ---> Removed intermediate container a6402ec91a05
 ---> 2604cf97a42b
Step 13/53 : ADD https://api.github.com/repos/gazebo-tooling/gzdev/git/refs/heads/master version.json

failed to get destination image "sha256:2604cf97a42bcab57a011610c16ca35019cacdf53e927cae3135e9006d6ab61c": image with reference sha256:2604cf97a42bcab57a011610c16ca35019cacdf53e927cae3135e9006d6ab61c was found but does not provide the specified platform (linux/arm)

gz-plugin4 4.0.0-2(ubuntu/resolute::armhf)

https://build.osrfoundation.org/job/gz-plugin4-debbuilder/379/

Step 12/53 : RUN echo "OSRF repositories configured: stable stable"
 ---> Running in 0068f70f37f2
OSRF repositories configured: stable stable
 ---> Removed intermediate container 0068f70f37f2
 ---> de44305e6e1e
Step 13/53 : ADD https://api.github.com/repos/gazebo-tooling/gzdev/git/refs/heads/master version.json

failed to get destination image "sha256:de44305e6e1ede592cb97eb710371b6a8b5f694a9a74310477d9431cfdf25694": image with reference sha256:de44305e6e1ede592cb97eb710371b6a8b5f694a9a74310477d9431cfdf25694 was found but does not provide the specified platform (linux/arm)