Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define isolated Auto Builder subscriptions for buildtools-prereqs repo #1210

Open
mthalman opened this issue Feb 29, 2024 · 1 comment
Open

Comments

@mthalman
Copy link
Member

The Auto Builder subscription for the dotnet/dotnet-buildtools-prereqs-docker is currently configured such that it handles updates for all Dockerfiles in the repo. This can cause a bottleneck when issues occur that may prevent images for other unaffected distros from getting published. For example, let's say that both Ubuntu and Mariner have base image updates that Auto Builder kicks off a build for. The build of the Ubuntu Dockerfiles happen to fail to some misconfiguration. But the Mariner Dockerfiles are unaffected and all pass their builds. This prevents Mariner from being updated until the Ubuntu issue is fixed.

The repo is already configured with OS-specific sub-manifests (example). So a way to help mitigate issues like this is to update the subscription to target those sub-manifests instead of the main manifest. Then the pipeline being targeted in the subscription can also be updated to the OS-specific pipeline. Having distinct pipelines is an important aspect of this since the Auto Builder logic bases its retry count on a per-pipeline basis. So if the pipeline fails to run 3 times in a row, it stops trying anymore. This would allow each OS to be handled independently wrt Auto Builder.

@mthalman
Copy link
Member Author

mthalman commented Mar 1, 2024

I ran a quick experiment to see if things would just work with a reconfigured subscription file. It doesn't just work.

Here's a sample entry from the config:

  {
    "manifest": {
      "owner": "dotnet",
      "repo": "dotnet-buildtools-prereqs-docker",
      "branch": "main",
      "path": "src/centos/manifest.json",
      "variables": {
        "FloatingTagSuffix": "",
        "UniqueId": ""
      }
    },
    "imageInfo": {
      "owner": "dotnet",
      "repo": "versions",
      "branch": "main",
      "path": "build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-main.json"
    },
    "pipelineTrigger": {
      "id": 437,
      "pathVariable": "imageBuilder.pathArgs"
    }
  }

The key here is that the manifest path is pointing to the manifest specific to CentOS.

It caused this exception:

Unhandled exception: System.IO.FileNotFoundException: Path specified in manifest file does not exist.
File name: 'src/centos/7/Dockerfile'
   at Microsoft.DotNet.ImageBuilder.ViewModel.ModelExtensions.ValidateFileReference(String path, String manifestDirectory) in /image-builder/src/ViewModel/ModelExtensions.cs:line 119
   at Microsoft.DotNet.ImageBuilder.ViewModel.ModelExtensions.ValidatePlatform(Platform platform, String manifestDirectory) in /image-builder/src/ViewModel/ModelExtensions.cs:line 163
   at Microsoft.DotNet.ImageBuilder.ViewModel.ModelExtensions.ValidateImage(Image image, String manifestDirectory) in /image-builder/src/ViewModel/ModelExtensions.cs:line 155
   at Microsoft.DotNet.ImageBuilder.ViewModel.ModelExtensions.ValidateRepo(Repo repo, String manifestDirectory) in /image-builder/src/ViewModel/ModelExtensions.cs:line 147
   at Microsoft.DotNet.ImageBuilder.ViewModel.ModelExtensions.Validate(Manifest manifest, String manifestDirectory) in /image-builder/src/ViewModel/ModelExtensions.cs:line 63
   at Microsoft.DotNet.ImageBuilder.ViewModel.ManifestInfo.Create(String manifestPath, ManifestFilter manifestFilter, IManifestOptionsInfo options) in /image-builder/src/ViewModel/ManifestInfo.cs:line 63
   at Microsoft.DotNet.ImageBuilder.ViewModel.ManifestInfo.Load(IManifestOptionsInfo options) in /image-builder/src/ViewModel/ManifestInfo.cs:line 46
   at Microsoft.DotNet.ImageBuilder.SubscriptionHelper.GetSubscriptionManifest(Subscription subscription, ManifestFilterOptions filterOptions, IGitService gitService, Action`1 configureOptions) in /image-builder/src/SubscriptionHelper.cs:line 78
   at Microsoft.DotNet.ImageBuilder.SubscriptionHelper.GetSubscriptionManifests(String subscriptionsPath, ManifestFilterOptions filterOptions, IGitService gitService, Action`1 configureOptions) in /image-builder/src/SubscriptionHelper.cs:line 34
   at Microsoft.DotNet.ImageBuilder.Commands.CopyBaseImagesCommand.ExecuteAsync() in /image-builder/src/Commands/CopyBaseImagesCommand.cs:line 62

I'm guessing what's happening is that it's trying to load the Dockerfile relative to the manifest file. But the Dockerfile paths in the manifest are relative to the repo root.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog
Development

No branches or pull requests

1 participant