When manifest list publishing fails, the images/manifest lists are left in a half-published state

[lbussell]

It is possible for image publishing to fail after copying images to the public ACR, and before creating and publishing manifest lists.

This causes a build to be left in a half-published state, where:

• individual images have been published
• manifest lists have not been published
• no publish notification was posted
• EOL annotations were not updated
• image info was not updated

This most commonly happens when filtering out images that are part of a manifest list tag. Here is an example build of dotnet-framework-docker where this happened: https://dev.azure.com/dnceng/internal/_build/results?buildId=2694908&view=results

We should re-arrange the publishing steps in the following order to prevent this from happening:

1. create manifest lists
2. push manifest lists to staging location (e.g. build-staging/<build ID>/dotnet/...)
3. copy all images to public acr location
4. copy all manifest lists to public acr location
5. proceed as usual with the rest of publishing

This would ensure that we fail the build early, before publishing any images, if there is a misconfiguration with manifest lists.

Related:

• Account for missing ACR images for build that only affects a portion of a shared tag #1511
• publishManifest command should skip unchanged tags #1510

[mthalman]

This is a bit tricky in those situations where a partial set of the manifest list is being published, which is ultimately the scenario that this is intended for. In that scenario the logic needs to know that it never built and staged a certain subset of the manifest list. So we actually need to pull down that set of missing images to the staging location from MAR. That will then allow you to construct the manifest list, targeting the full set of images in the staging location.

[lbussell]

I don't want to get this mixed up with #1511. My point is that if anything at all goes wrong in manifest list creation, then the build will be half-published which is bad. If anything goes wrong in manifest list creation, I would like the build to fail before pushing any images. That is what this issue is about.

Supporting situations where a partial set of the manifest list is being published should be covered by #1511.

[lbussell]

I updated the title to hopefully make the intention a little clearer.