Skip to content

Migrate to Ubuntu 24.04#21761

Draft
cho-m wants to merge 2 commits intomainfrom
move-to-ubuntu24.04
Draft

Migrate to Ubuntu 24.04#21761
cho-m wants to merge 2 commits intomainfrom
move-to-ubuntu24.04

Conversation

@cho-m
Copy link
Copy Markdown
Member

@cho-m cho-m commented Mar 18, 2026

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) with your changes locally?

  • AI was used to generate or assist with generating this PR. Please specify below how you used AI to help you, and what steps you have taken to manually verify the changes.

Draft until we get closer to brew 5.2.0 release date.

Related work:

Homebrew/core PRs blocked by Ubuntu 22.04:

Formulae with LLVM Clang workarounds to revert after migrations:

  • apache-arrow
  • gnome-builder
  • mdk
  • mysql
  • opentelemetry-cpp
  • protobuf
  • qxmpp

Formulae with GCC dependency to rebuild after migration:

  • btop (?)
  • ladybug
  • libunicode
  • vroom

@cho-m cho-m added in progress Maintainers are working on this release blocker A new release should not be done until this is fixed. linux labels Mar 18, 2026
@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 18, 2026

Failures:

  • formula audit runs on "ghcr.io/homebrew/brew:main" which is still Ubuntu 22.04 until PR is merged. Could either switch to "ghcr.io/homebrew/ubuntu24.04:main" or split docker images to separate PR.
  • Bookworm (old stable) is on GCC 12. Not sure yet if there is a Linux distro running glibc < 2.39 and has libstdc++ for gcc >= 14. May need to find a way to ignore that audit failure.
  • Docker image for 22.04 may need to be dropped if brew doctor cannot be ignored. Can add 26.04 pre-release image instead https://hub.docker.com/_/ubuntu#supported-tags-and-respective-dockerfile-links

@MikeMcQuaid
Copy link
Copy Markdown
Member

  • formula audit runs on "ghcr.io/homebrew/brew:main" which is still Ubuntu 22.04 until PR is merged. Could either switch to "ghcr.io/homebrew/ubuntu24.04:main"

@cho-m this makes sense to me 👍🏻.

  • Bookworm (old stable) is on GCC 12. Not sure yet if there is a Linux distro running glibc < 2.39 and has libstdc++ for gcc >= 14. May need to find a way to ignore that audit failure.

Which audit failure specifically? All we're trying to do in this case is ensure that the glibc formula is installed if needed. I do think we should aim to find another distro for this.

It looks from 👇🏻 that Ubuntu 22.04 may be enough for that?

Yes, we should drop it and bump to versions that don't need to install glibc such as yeh the 26.04 image.


At least some of this we can do before 5.2.0.

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 19, 2026

Which audit failure specifically? All we're trying to do in this case is ensure that the glibc formula is installed if needed. I do think we should aim to find another distro for this.

The CI / test-bot run failed due to https://github.com/Homebrew/brew/actions/runs/23263094267/job/67636951864?pr=21761#step:9:73

  libfaketime
    * Formulae in homebrew/core should not have a Linux-only dependency on GCC.

Maybe that audit itself shouldn't fail for :implicit dependency? Will experiment with this locally.

EDIT: If needed, RHEL 9-based distros (e.g. Rocky 9) use glibc 2.34 + GCC 14 and will be supported for a long time.

@MikeMcQuaid
Copy link
Copy Markdown
Member

EDIT: If needed, RHEL 9-based distros (e.g. Rocky 9) use glibc 2.34 + GCC 14 and will be supported for a long time.

Don't really care about support. If we can use e.g. Ubuntu that we've already used for this test: all the better.

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented Mar 19, 2026

  • Docker image for 22.04 may need to be dropped if brew doctor cannot be ignored

We had an overlap last time so should just be able to adjust it. Given its usage levels, it would be ideal to not have to skip deprecation just because of a brew doctor warning we created. There's several brew doctor warnings we skip in test bot, so can easily do that here too.

Promoting homebrew/brew image more may help this in the future. Maybe we even stop doing any more versioned ubuntu* images.

  • Bookworm (old stable) is on GCC 12. Not sure yet if there is a Linux distro running glibc < 2.39 and has libstdc++ for gcc >= 14. May need to find a way to ignore that audit failure.

The point of the test is to test distros on older GCC and Glibc. The fact it's even on Bookworm is actually a mistake.

Maybe that audit itself shouldn't fail for :implicit dependency?

Yeah, this is the fix to make here. It should not be failing an audit on something that's implicitly added.

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 19, 2026

The point of the test is to test distros on older GCC and Glibc. The fact it's even on Bookworm is actually a mistake.

Debian oldstable does change with every new release (2 years?). Can switch it to a fixed major version, maybe Ubuntu 22.04 or Debian bookworm assuming Docker Hub image isn't planned for removal.

Yeah I think this is the fix. It should not be failing an audit on something that's implicitly added.

Can do next run with 22.04 restored as fix should be in.

@MikeMcQuaid
Copy link
Copy Markdown
Member

Given its usage levels, it would be ideal to not have to skip deprecation just because of a brew doctor warning we created. There's several brew doctor warnings we skip in test bot, so can easily do that here too.

We should just deprecate it ASAP in this case. We aren't strictly bound to our own deprecation process, particularly on Docker images where we've been more flexible in the past and no-one complained.

maybe Ubuntu 22.04

This seems ideal.


Honestly I'm not even sure why we're publishing as many Docker images as we do. The images we create are trivial to replicate and build elsewhere and they just seem to cause us a decent amount of grief with both our CI (they are flaky) and the deprecation process.

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented Mar 20, 2026

Honestly I'm not even sure why we're publishing as many Docker images as we do

Exactly why I suggested maybe not doing any more versioned ubuntu* images.

Ultimately, we set up tap-new to output ubuntu22.04 images which is in part why they are widely used right now and ideally need a deprecation period. If we avoid doing that in the future, then we can drop these things easier.

@MikeMcQuaid
Copy link
Copy Markdown
Member

Exactly why I suggested maybe not doing any more versioned ubuntu* images.

Ultimately, we set up tap-new to output ubuntu22.04 images which is in part why they are widely used right now and ideally need a deprecation period. If we avoid doing that in the future, then we can drop these things easier.

Fair, good point. I'd be game to kick off that deprecation period literally as soon as we switch things over now.

@Bo98 remind me (and perhaps help @cho-m): how we did/should do deprecation of the previous images?

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 20, 2026

Ultimately, we set up tap-new to output ubuntu22.04 images which is in part why they are widely used right now and ideally need a deprecation period. If we avoid doing that in the future, then we can drop these things easier.

I did update this in

I verified it worked for a new tap I created (bottles built, commit pushed, release with bottles, etc.). Not sure if there are any edge cases that need to handled

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented Mar 20, 2026

@Bo98 remind me (and perhaps help @cho-m): how we did/should do deprecation of the previous images?

18.04: b34253c

Or the 20.04 deprecation in the newer workflow layout: c9a7b62 (also drops 18.04 so ignore those changes)

@cho-m cho-m force-pushed the move-to-ubuntu24.04 branch 3 times, most recently from a90f635 to 02015aa Compare March 20, 2026 21:42
@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 20, 2026

Looks like formula audit and test-bot jobs may still need a versioned container as rolling label will fetch old Ubuntu and trigger glibc error.

Haven't thought of any great alternative:

  • ignoring glibc error during migration would allow test to pass, but this means we aren't verifying newer Ubuntu before PR merge
  • could add a rolling label for "next" ubuntu and have some logic to pick it during migration PR

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented Mar 20, 2026

Audit will be fixed after the homebrew-core PR merge won't it?

For test bot, I would perhaps wonder why we even need a container for them (besides Homebrew glibc test) - does it really need Homebrew/core tapped?

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 20, 2026

Audit will be fixed after the homebrew-core PR merge won't it?

Yeah. I misread the error.

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 20, 2026

For test bot, I would perhaps wonder why we even need a container for them (besides Homebrew glibc test) - does it really need Homebrew/core tapped?

Not familiar with test but container may simplify setup. We need system packages for build requirements and then --test-default-formula builds core formulae libfaketime and xz

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented Mar 20, 2026

Oh did not realise that changed to use Homebrew/core formula, we used to have a test fixture for exactly that purpose.

@MikeMcQuaid
Copy link
Copy Markdown
Member

For test bot, I would perhaps wonder why we even need a container for them (besides Homebrew glibc test) - does it really need Homebrew/core tapped?

Yes, we're trying to replicate the homebrew/core CI flow as closely as possible to blow up here before we blow up there.

as rolling label will fetch old Ubuntu and trigger glibc error.

Debian oldstable test-bot triggers this error so we can similarly handle it for old Ubuntu as we have for that.

Copy link
Copy Markdown
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks!

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 21, 2026

To be specific, the glibc problem is for our "latest" tests, i.e.

- name: test-bot (Linux x86_64)
runs-on: ubuntu-latest
container: ghcr.io/homebrew/brew:main

As we haven't built the new container yet by the time this PR runs.

@MikeMcQuaid
Copy link
Copy Markdown
Member

As we haven't built the new container yet by the time this PR runs.

Ah, gotcha. Fine to comment that out in this PR and then change it in the next. Ideally we'd have a better flow than that, though.

@cho-m cho-m force-pushed the move-to-ubuntu24.04 branch from 02015aa to 9a83046 Compare March 25, 2026 20:58
@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 25, 2026

We still have ubuntu24.04 image so switched back to that in tests.yml. Will need to figure out a better alternative for Ubuntu 26.04 if we want to get rid of these extra images.

Also updated Support Tiers documentation for glibc versions. I do want to find a way to use templates for these so we can avoid having to update versions manually. However, not essential as documentation can always be updated separately if we forget to handle in main PR.

Tempted to also remove default version from Dockerfile and make docker.yml workflow run brew to get version number to reduce total files we need to update. Ideally only need 1 file (either os.rb or moving it to a json/yaml file)

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 25, 2026

Looks like Ubuntu apt install will hit tzdata prompt unlike Debian. May be some difference in setup or DEBIAN_FRONTEND default.

EDIT: also need

cho-m and others added 2 commits March 25, 2026 22:17
@cho-m cho-m force-pushed the move-to-ubuntu24.04 branch from c7dead1 to 2fc003a Compare March 26, 2026 02:18
@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 26, 2026

I guess we don't actually need Homebrew/core checked out given CI is running on JSON API. (EDIT: tap used for brew audit but not brew install)

Looks like GitHub runner availability is limited right now as API generation is happening every hour which led to a couple run failures.


PR is ready for whenever we want to prepare 5.2.0 release. Can un-draft it then.

Copy link
Copy Markdown
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @cho-m! Looks good for next release and that everything that can be done now/extracted in advance has been.

May be worth documenting this PR (in itself or another PR 🤔) inside the Linux-CI.md page as an example of what will need done to bump in future.

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 26, 2026

May be worth documenting this PR (in itself or another PR 🤔) inside the Linux-CI.md page as an example of what will need done to bump in future.

Sure. I've been jotting some notes down locally. Will edit them and open PR to add to docs.


Also, just remembered we'll need to handle self-hosted too - https://github.com/Homebrew/actions/blob/main/create-gcloud-instance/create-gcloud-instance.sh#L8.

Can keep this as a manual update given we may replace GCP. Once we figure out new self-hosted, adding a way to sync Ubuntu version number rather than hardcoding will simplify maintenance.

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented Mar 26, 2026

Also, just remembered we'll need to handle self-hosted too - https://github.com/Homebrew/actions/blob/main/create-gcloud-instance/create-gcloud-instance.sh#L8.

Can keep this as a manual update given we may replace GCP. Once we figure out new self-hosted, adding a way to sync Ubuntu version number rather than hardcoding will simplify maintenance.

What date is 5.2.0? Migration of all that stuff might happen in April, only just finished the 10.15 & Node 24 stuff this week so Linux should be next up.

But even if it's before, it's a one line change so let's keep it simple rather than messing around with syncing.

@cho-m
Copy link
Copy Markdown
Member Author

cho-m commented Mar 26, 2026

What date is 5.2.0? Migration of all that stuff might happen in April, only just finished the 10.15 & Node 24 stuff this week so Linux should be next up.

I don't think we have a planned date yet so migration may happen before release.

But even if it's before, it's a one line change so let's keep it simple rather than messing around with syncing.

Makes sense. GCP also changed name from ubuntu-2204-lts to ubuntu-2404-lts-amd64 so not as simple to sync without another constant.

@MikeMcQuaid
Copy link
Copy Markdown
Member

But even if it's before, it's a one line change so let's keep it simple rather than messing around with syncing.

Would be nice to avoid a one line change that's outside this PR as these things tend to get forgotten.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

in progress Maintainers are working on this linux release blocker A new release should not be done until this is fixed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants