ci: use cachix for all nix jobs#15327
Open
Alizter wants to merge 1 commit into
Open
Conversation
Member
|
How big of a cache do we need? I've bumped our github actions cache to 50GB previously. Wasn't that enough? |
Collaborator
Author
It was causing cache stores to be rejected due to meeting billing limitations. Its opaque to me why this is happening. I'm able to give a cachix secret here in the repo allowing us to use a cachix server which I've been testing out successfully in my fork. |
Collaborator
Author
|
Here is an example of such an issue: https://github.com/ocaml/dune/actions/runs/28181786547/job/83473615172?pr=15326#step:9:4954 This has been happening regularly. |
Replace the per-job nix-community/cache-nix-action setup with a single
cachix/cachix-action step that pushes/pulls from the ocaml-dune cache.
Also adds cachix to nix jobs that previously had no caching at all
(nix-build, nix-test, fmt, doc, bootstrap, build-microbench).
The GitHub Actions cache backend repeatedly hit its spending limit
('Cache reservation failed: ...read only to prevent additional
charges'), at which point new saves failed and restores fell back to
stale prefix matches. cache-nix-action also accumulated one full-sized
entry per distinct '**/*.nix'/flake.lock hash, since it never purged
old prefix-matched entries on save. Cachix has no per-repo budget,
shares across branches and PRs, and avoids the duplicate-cache
problem.
Signed-off-by: Ali Caglayan <alizter@gmail.com>
Alizter
force-pushed
the
push-lvxmwwsrvtsu
branch
from
6b6d4ce to
6cfc504
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
nix-community/cache-nix-action@v7withcachix/cachix-action@v15(cache nameocaml-dune, authed viaCACHIX_AUTH_TOKEN_OCAML_DUNE) on all six nix jobs that had it:nix-build-4-14,nix-build-ox,rocq,rocq-native,nix-oxcaml,utop-dev-tool-test.nix-build,nix-test,fmt,doc,bootstrap,build-microbench.Why
The GitHub Actions cache backend repeatedly hit its per-repo spending limit (`Cache reservation failed: You have reached your configured budget, your cache is now read only to prevent additional charges.`). Once the budget was exhausted,
cache-nix-actionsaves silently failed and subsequent runs fell back viarestore-prefixes-first-matchto whatever stale entry was still around — which is whynix-oxcamlon main kept restoring a 626 MiB pre-#15299 cache and re-downloading the OxCaml closure on every run.Independently,
cache-nix-actionnever purged old prefix-matched entries on save, so each distinct**/*.nix/flake.lockhash accumulated a full-sized entry. A snapshot before this PR showed ~3.6 GiB of duplicaterocq/rocq-native/nix-build-4-14caches alone.Cachix has no per-repo budget, is shared across branches and PRs, and avoids both problems.
Test plan
ocaml-dunecachix cache.*.nix) shows cachix hits in the logs and fasternix developstartup for the heavier jobs (nix-oxcaml,nix-build-ox,rocq*).