brew install mesa fails at link step on Fedora Silverblue (probably not a formula bug)

[averms]

brew doctor output

Before installing mesa: Your system is ready to brew.

After:

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  mesa

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.2.14
ORIGIN: https://github.com/Homebrew/brew
HEAD: f6a4f42cc346694fac94fa1643bc2809d383b6f1
Last commit: 4 days ago
Core tap JSON: 23 Mar 21:35 UTC
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :0
HOMEBREW_EDITOR: /usr/bin/nano
HOMEBREW_MAKE_JOBS: 2
HOMEBREW_TEMP: /var/tmp
Homebrew Ruby: 3.1.4 => /var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/bin/ruby
CPU: dual-core 64-bit zen3
Clang: 17.0.6
Git: 2.44.0 => /bin/git
Curl: 8.2.1 => /bin/curl
Kernel: Linux 6.7.9-200.fc39.x86_64 x86_64 GNU/Linux
OS: Fedora release 39 (Thirty Nine)
Host glibc: 2.38
/usr/bin/gcc: 13.2.1
/usr/bin/ruby: N/A
glibc: N/A
gcc@11: N/A
gcc: N/A
xorg: N/A

What were you trying to do (and why)?

I was trying to install mesa.

What happened (include all command output)?

The interesting part of te_log.log is reproduced below:

==> Pouring mesa--24.0.3.x86_64_linux.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /home/linuxbrew/.linuxbrew
Could not symlink ../../../../../../../home/linuxbrew/.linuxbrew/Cellar/libvdpau/1.5/lib/vdpau/libvdpau_trace.so
Target /home/linuxbrew/.linuxbrew/Cellar/libvdpau/1.5/lib/vdpau/libvdpau_trace.so
is a symlink belonging to libvdpau. You can unlink it:
  brew unlink libvdpau

To force the link and overwrite all conflicting files:
  brew link --overwrite libvdpau

To list all files that would be deleted:
  brew link --overwrite libvdpau --dry-run

Possible conflicting files are:

What did you expect to happen?

No link error.

Step-by-step reproduction instructions (by running brew commands)

1. Spin up a Fedora Silverblue VM.
2. Install the Homebrew on Linux requirements. There is no "groupinstall" in
   `rpm-ostree` and the other mentioned packages are already included so I just ran

   rpm-ostree install git gcc

3. Reboot.
4. Install Homebrew and run the shell startup code.
5. Raise the limit on maximum open files with `ulimit -n 99999`. 
6. Change the temp directory to `var/tmp` with `export HOMEBREW_TEMP=/var/tmp`.
7. Verify that `brew doctor` output is good.
8. Install mesa with `brew install mesa`

For clarification, step 5 avoids a "too many open files" error like in #11530 and step 6 avoids a "no space left on device" error like in Linuxbrew/brew#923.

[averms]

Here is the full stacktrace:

Error: Could not symlink ../../../../../../../home/linuxbrew/.linuxbrew/Cellar/libvdpau/1.5/lib/vdpau/libvdpau_trace.so
Target /home/linuxbrew/.linuxbrew/Cellar/libvdpau/1.5/lib/vdpau/libvdpau_trace.so
is a symlink belonging to libvdpau. You can unlink it:
  brew unlink libvdpau

To force the link and overwrite all conflicting files:
  brew link --overwrite libvdpau

To list all files that would be deleted:
  brew link --overwrite libvdpau --dry-run
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/pathname.rb:330:in `symlink'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/pathname.rb:330:in `make_relative_symlink'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/extend/pathname.rb:563:in `make_relative_symlink'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:593:in `make_relative_symlink'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:645:in `block in link_dir'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/pathname.rb:571:in `block in find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:49:in `block (2 levels) in find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:48:in `catch'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:48:in `block in find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:43:in `each'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:43:in `find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/pathname.rb:571:in `find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:622:in `link_dir'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:566:in `resolve_any_conflicts'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:661:in `block in link_dir'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/pathname.rb:571:in `block in find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:49:in `block (2 levels) in find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:48:in `catch'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:48:in `block in find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:43:in `each'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/find.rb:43:in `find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/lib/ruby/3.1.0/pathname.rb:571:in `find'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:622:in `link_dir'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:439:in `link'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/link.rb:112:in `block (2 levels) in link'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:329:in `block in lock'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/lock_file.rb:35:in `with_lock'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/keg.rb:324:in `lock'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/link.rb:107:in `block in link'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/link.rb:57:in `each'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/cmd/link.rb:57:in `link'
/var/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:86:in `<main>'

Pure speculation but I wonder if this is related to Silverblue symlinking /home to var/home

[Bo98]

Pure speculation but I wonder if this is related to Silverblue symlinking /home to var/home

This seems probable. Though symlink target being in the Cellar itself seems very odd and I'm not entirely sure what's so special about that particular file only?

Usually the error would be something like:

Error: Could not symlink lib/vdpau/libvdpau_trace.so
Target /home/linuxbrew/.linuxbrew/lib/vdpau/libvdpau_trace.so

[averms]

and I'm not entirely sure what's so special about that particular file only?

Actually, linking the other way around (mesa before libvdpau) gives the same symlink error but on a different file. Running brew unlink mesa libvdpau && brew link mesa libvdpau outputs the following:

Unlinking /home/linuxbrew/.linuxbrew/Cellar/mesa/24.0.3... 0 symlinks removed.
Unlinking /home/linuxbrew/.linuxbrew/Cellar/libvdpau/1.5... 6 symlinks removed.
Linking /home/linuxbrew/.linuxbrew/Cellar/mesa/24.0.3... 75 symlinks created.
Linking /home/linuxbrew/.linuxbrew/Cellar/libvdpau/1.5... 
Error: Could not symlink ../../../../../../../home/linuxbrew/.linuxbrew/Cellar/mesa/24.0.3/lib/vdpau/libvdpau_nouveau.so
Target /home/linuxbrew/.linuxbrew/Cellar/mesa/24.0.3/lib/vdpau/libvdpau_nouveau.so
is a symlink belonging to mesa. You can unlink it:
  brew unlink mesa

To force the link and overwrite all conflicting files:
  brew link --overwrite mesa

To list all files that would be deleted:
  brew link --overwrite mesa --dry-run

[Bo98]

Hmm I wonder if it gets mixed up with handling the vdpau subdirectory in /home/linuxbrew/.linuxbrew/lib.

The correct behaviour is when the first formula is installed then that's a whole directory symlink, but when the second formula is installed it recognises that, makes it a regular directory and then symlinks the contents.

Definitely works correctly on Ubuntu so is probably some /var/home incorrect handling.

I don't have a Silverblue setup on hand, though I'll look and see if there's a Docker image with a similar filesystem layout.

[cho-m]

Maybe also check output of brew link mesa --dry-run --verbose as it should show commands brew is attempting.

---

I'm guessing it could be handling of link where some parts of code directly use HOMEBREW_PREFIX/... while others run .realpath or .resolved_path. Also possible that some symlink? check on HOMEBREW_PREFIX-rooted paths will break.

---

EDIT: Though it looks like some support went into #15656. May need to be debugged further by anyone who has Silverblue

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[averms]

Yeah it's definitely the /home -> var/home symlink.

I don't have a Silverblue setup on hand, though I'll look and see if there's a Docker image with a similar filesystem layout.

Try the following Dockerfile:

FROM debian:12
RUN apt-get -y update && apt-get -y install \
    curl \
    git \
    gcc
RUN rm -r /home && mkdir /var/home && ln -s var/home /home
RUN useradd --create-home linuxbrew
USER linuxbrew
WORKDIR /home/linuxbrew
RUN /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
ENV HOMEBREW_TEMP=/var/tmp
RUN .linuxbrew/bin/brew shellenv >>.bashrc

Just enter the container and try to install mesa.