Skip to content

[storage/mmr] change pruning API to be location instead of position based#3309

Open
roberto-bayardo wants to merge 2 commits intomainfrom
prune-api
Open

[storage/mmr] change pruning API to be location instead of position based#3309
roberto-bayardo wants to merge 2 commits intomainfrom
prune-api

Conversation

@roberto-bayardo
Copy link
Collaborator

@roberto-bayardo roberto-bayardo commented Mar 5, 2026

Changes the MMR pruning API to align the pruning boundary to a leaf (Location), and to return Error on bad input instead of panicking (since it's pub). This is more consistent with how proving works (over locations). Also changes bounds() to be location based since begin/end are now always leaf aligned.

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 5, 2026

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
commonware-mcp 8d605a9 Mar 07 2026, 04:09 AM

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Mar 5, 2026

Deploying monorepo with  Cloudflare Pages  Cloudflare Pages

Latest commit: 8d605a9
Status: ✅  Deploy successful!
Preview URL: https://f508ffd0.monorepo-eu0.pages.dev
Branch Preview URL: https://prune-api.monorepo-eu0.pages.dev

View logs

@roberto-bayardo roberto-bayardo requested review from Copilot and danlaine and removed request for danlaine March 5, 2026 19:27
@roberto-bayardo roberto-bayardo moved this to In Progress in Tracker Mar 5, 2026
@roberto-bayardo roberto-bayardo added breaking-format This PR modifies codec and/or storage formats. breaking-api This PR modifies the public interface of a function. labels Mar 5, 2026
@roberto-bayardo roberto-bayardo force-pushed the prune-api branch 2 times, most recently from ac621cf to 2466678 Compare March 5, 2026 19:44
@roberto-bayardo roberto-bayardo marked this pull request as ready for review March 5, 2026 19:45
@roberto-bayardo roberto-bayardo moved this from In Progress to Ready for Review in Tracker Mar 5, 2026
@roberto-bayardo roberto-bayardo force-pushed the prune-api branch 2 times, most recently from 64385ba to a3fe75c Compare March 5, 2026 19:55
@roberto-bayardo roberto-bayardo requested a review from danlaine March 5, 2026 19:58
@roberto-bayardo
Copy link
Collaborator Author

bugbot run

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@roberto-bayardo roberto-bayardo force-pushed the prune-api branch 4 times, most recently from 50c06b8 to 4dfe3b0 Compare March 7, 2026 04:00
@codecov
Copy link

codecov bot commented Mar 7, 2026

Codecov Report

❌ Patch coverage is 97.70115% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.26%. Comparing base (ead4a09) to head (8d605a9).

Files with missing lines Patch % Lines
storage/src/mmr/mem.rs 93.87% 1 Missing and 2 partials ⚠️
storage/src/mmr/journaled.rs 99.09% 1 Missing ⚠️
@@           Coverage Diff           @@
##             main    #3309   +/-   ##
=======================================
  Coverage   93.25%   93.26%           
=======================================
  Files         423      423           
  Lines      146470   146499   +29     
  Branches     3455     3458    +3     
=======================================
+ Hits       136597   136626   +29     
  Misses       8779     8779           
  Partials     1094     1094           
Files with missing lines Coverage Δ
storage/src/bitmap/authenticated.rs 90.62% <100.00%> (-0.02%) ⬇️
storage/src/journal/authenticated.rs 95.09% <100.00%> (-0.01%) ⬇️
storage/src/mmr/batch.rs 97.71% <100.00%> (ø)
storage/src/mmr/proof.rs 97.59% <100.00%> (ø)
storage/src/qmdb/any/ordered/fixed.rs 97.88% <100.00%> (-0.01%) ⬇️
storage/src/qmdb/current/db.rs 88.99% <100.00%> (-0.03%) ⬇️
storage/src/qmdb/current/grafting.rs 90.61% <ø> (ø)
storage/src/mmr/journaled.rs 97.97% <99.09%> (+0.02%) ⬆️
storage/src/mmr/mem.rs 98.61% <93.87%> (-0.22%) ⬇️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ead4a09...8d605a9. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

let config = Config {
nodes,
pruned_to_pos: Position::new(input.pruned_to_pos),
pruned_to: Location::new(input.pruned_to_pos),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to change the name of of pruned_to_pos?

hasher,
)?;
let prune_pos = Position::new(effective_prune_pos);
let prune_pos = effective_prune_pos;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be a new var?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLM claims an issue:

• The patch mostly updates pruning APIs from positions to locations, but it introduces one data-loss path in init_sync() for non-leaf boundaries and weakens mem::Mmr::prune()'s documented error handling for invalid locations. Those issues make the patch unsafe to consider fully correct.

  Full review comments:

  - [P2] Validate sync boundary before clearing persisted data — /Users/patrickogrady/code/monorepo-pr-3309/storage/src/mmr/journaled.rs:409-410
    If init_sync() is called with a non-leaf SyncConfig.range.start and the existing journal is empty or still behind that boundary, this branch calls clear_to_size(*cfg.range.start) before the later Location::try_from(cfg.range.start) validation runs. The function then returns an error
    after it has already truncated/reinitialized the persisted journal, so a caller using a node-position boundary (which the public Range<Position> API still permits) can lose on-disk data on a failed sync attempt.
  - [P3] Check prune input validity before returning LeafOutOfBounds — /Users/patrickogrady/code/monorepo-pr-3309/storage/src/mmr/mem.rs:331-334
    If a caller passes Location::new(u64::MAX) or any other value above MAX_LOCATION, this branch returns LeafOutOfBounds as soon as the tree has fewer leaves than that value, because the Position::try_from(loc) validation now happens afterwards. That breaks the documented LocationOverflow
    contract of Mmr::prune() and makes malformed locations indistinguishable from an ordinary out-of-range prune request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-api This PR modifies the public interface of a function. breaking-format This PR modifies codec and/or storage formats.

Projects

Status: Ready for Review

Development

Successfully merging this pull request may close these issues.

2 participants