Skip to content

[storage/journal] Make Contiguous Variable #1718

@patrick-ogrady

Description

@patrick-ogrady

Unlike journal::fixed, journal::variable does not assume that you are working over a contiguous global log (just one per section). The rationale for this was that you need to store consensus messages in multiple views and you want to prune them independently (if you added to a contiguous log, you'd have to track the lowest view active at some index):

async fn prune_views(&mut self) {
// Get last min
let min = min_active(self.activity_timeout, self.last_finalized);
let mut pruned = false;
loop {
// Get next key
let next = match self.views.keys().next() {
Some(next) => *next,
None => return,
};
// If less than min, prune
if next >= min {
break;
}
self.views.remove(&next);
debug!(
view = next,
last_finalized = self.last_finalized,
"pruned view"
);
pruned = true;
}
// Prune journal up to min
if pruned {
self.journal
.as_mut()
.unwrap()
.prune(min)
.await
.expect("unable to prune journal");
}
// Update metrics
self.tracked_views.set(self.views.len() as i64);
}

We should make a wrapper around journal::variable that provides this contiguous functionality (likely from logic that already exists in adb). This would make implementing much of the adb-related logic MUCH simpler (as there is a ton of logic mapping index to location).

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

Status

Staged

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions