brew install llvm fails with "Cannot write: No space left on device"

[tenturestept]

Describe the bug

I installed Homebrew on a fresh Bluefin instance with ujust brew. I tried to install LLVM with brew install --verbose llvm. However, it failed with the following output:

==> Fetching llvm
==> Downloading https://ghcr.io/v2/homebrew/core/llvm/blobs/sha256:7ccc6f7936a7b1ee36403822c3b301264aa37956bb5a3c21eb85c3466ef36265
Already downloaded: /var/home/testuser/.cache/Homebrew/downloads/15a0768d9feadddd145a0d039436794d24fe308b87c82227cc6c1650c38d3311--llvm--17.0.6_1.x86_64_linux.bottle.tar.gz
==> Verifying checksum for '15a0768d9feadddd145a0d039436794d24fe308b87c82227cc6c1650c38d3311--llvm--17.0.6_1.x86_64_linux.bottle.tar.gz'
==> Pouring llvm--17.0.6_1.x86_64_linux.bottle.tar.gz
/usr/bin/env tar --extract --no-same-owner --file /var/home/testuser/.cache/Homebrew/downloads/15a0768d9feadddd145a0d039436794d24fe308b87c82227cc6c1650c38d3311--llvm--17.0.6_1.x86_64_linux.bottle.tar.gz --directory /tmp/homebrew-unpack20240331-16896-oen4qe
tar: llvm/17.0.6_1/lib/libclangStaticAnalyzerCheckers.a: Wrote only 9216 of 10240 bytes
tar: llvm/17.0.6_1/lib/libclangStaticAnalyzerCore.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangStaticAnalyzerFrontend.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangSupport.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidy.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyAbseilModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyAlteraModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyAndroidModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyBoostModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyBugproneModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyCERTModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyConcurrencyModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyCppCoreGuidelinesModule.a: Cannot write: No space left on device
tar: llvm/17.0.6_1/lib/libclangTidyDarwinModule.a: Cannot write: No space left on device
...
# there are many more lines of this

What did you expect to happen?

I expected llvm to install without any errors.

Output of rpm-ostree status

State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: no runs since boot
Deployments:
● ostree-image-signed:docker://ghcr.io/ublue-os/bluefin:gts
                   Digest: sha256:4c28601aad6f70732d81376fd96679c9e8c5af481c84e0c87f06a21724afe067
                  Version: 38.20240330.0 (2024-03-30T16:47:53Z)

  ostree-image-signed:docker://ghcr.io/ublue-os/bluefin:gts
                   Digest: sha256:f1590675d4602ce826e11b10e0a0ceddb3abac54fb70dac13c83467d058e95a9
                  Version: 38.20240304.0 (2024-03-05T04:29:10Z)

Extra information or context

I found an old issue where someone suggested setting the HOMEBREW_TEMP environment variable to /var/tmp. This fixed the issue for me.

The workaround on our part is pretty simple but I'm wondering if this should be raised on the Homebrew repo?

[castrojo]

I'm not sure if the workaround should live here or in homebrew, let's see what the other's say - perhaps we can ship it with that variable set ootb?

[tenturestept]

also right after i fixed this i encountered another issue that was already reported in Homebrew/brew#16943.

[m2Giles]

The original problem is that /tmp is a tmpfs and you ran out of space on the device (likely out of backing ram)

As it said, you should use /var/tmp for builds that will exceed the space you have allocated for /tmp. Moving to /var/tmp would not be ideal since we lose out of working out of ram. Most operations won't exhaust ram.

[tenturestept]

yeah llvm is a pretty big bottle, most will probably be less than 8 gigs. however, what is the benefit of untarring a bottle to ram? won't it be moved anyway so i don't think you will save on ssd writes.

[m2Giles]

tmpfs is faster. So instead of untarring on disk and then moving to correct location on disk, untar in ram and then move into place. Since most bottles will be smaller and most actions will be small, this results in a speed improvement. It's not to save ssd writes (though that is a benefit), it's simply to do the untar action faster.