Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse milhouse subtrees to shrink inactivity_scores in memory #7062

Merged
merged 3 commits into from
Mar 3, 2025

Conversation

michaelsproul
Copy link
Member

@michaelsproul michaelsproul commented Mar 3, 2025

Use a new Milhouse method called intra_rebase to reuse identical subtrees in inactivity_scores. This reduces the memory used by each inactivity_scores update from ~70MB+ to ~4-5MB.

To update to milhouse's main branch I needed to steal Paul's changes from this PR, which I squashed into a single commit to make it easier to remove unwanted changes from unstable:

I tested this change using an even hackier branch here, which includes the Milhouse mem-usage changes: https://github.com/michaelsproul/lighthouse/tree/milhouse-mem-size-holesky-rescue

Command for testing:

RUST_LOG=debug lcli skip-slots --pre-state-path state_3714912.ssz --slots 32 --network holesky

Results:

Inactivity score intra-rebase time: 47ms
Post-state balances size (total/diff): 84594016-84594016 B/79744688 B
Post-state validators size: 522924016-522924016 B/9416 B
Post-state inactivity_scores size: 84596096-84596096 B/4696384 B

@michaelsproul michaelsproul requested a review from jxs as a code owner March 3, 2025 04:57
@michaelsproul michaelsproul changed the title Intra rebase Use milhouse intra_rebase to shrink inactivity_scores in memory Mar 3, 2025
@michaelsproul michaelsproul changed the title Use milhouse intra_rebase to shrink inactivity_scores in memory Reuse milhouse subtrees to shrink inactivity_scores in memory Mar 3, 2025
@michaelsproul michaelsproul added v7.0.0-beta.2 Second hacky Holesky rescue release optimization Something to make Lighthouse run more efficiently. labels Mar 3, 2025
@michaelsproul michaelsproul merged commit 6399ad4 into holesky-rescue Mar 3, 2025
22 of 28 checks passed
@michaelsproul michaelsproul deleted the intra-rebase branch March 3, 2025 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optimization Something to make Lighthouse run more efficiently. v7.0.0-beta.2 Second hacky Holesky rescue release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant