feat(persistence): implement RocksDB provider #20071

duyquang6 · 2025-12-02T10:40:36Z

Resolve #20005
Changes:

Apply some recommend tuning config from RocksDB docs, verified enabled by RocksDB log
Expose api: Get, Put, Delete, WriteBatch
Compatible with db-api non-dupsort Table
Add RocksDB metrics: count, latency
Add tests & verify

duyquang6 · 2025-12-02T15:00:12Z

RocksDB does not work for cross compile on Windows currently, specially using mingw toolchain. Might be msvc can work (worth to try)

I recommend temporarily disabling RocksDB support on Windows and using libmdbx as the fallback. Improving Windows compatibility for RocksDB can be considered future work, wdyt ?

error log: https://github.com/paradigmxyz/reth/actions/runs/19861648226/job/56913064304?pr=20071#step:8:21555

related issue:

mattsse

a few (early and pedantic) comments

mattsse · 2025-12-02T15:01:15Z

Cargo.toml

 toml = "0.8"

+# database
+rocksdb = { version = "0.24", default-features = false, features = ["bindgen-runtime", "lz4", "zstd", "jemalloc"] }


I think enabling these by default is okay

mattsse · 2025-12-02T15:03:25Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+    /// Puts a value into the specified table.
+    pub fn put<T: Table>(&self, key: T::Key, value: T::Value) -> ProviderResult<()> {
+        self.execute_metered(RocksDBOperation::Put, T::NAME, || {
+            let encoded_key = key.encode();


we likely also want put_encoded I think

mattsse · 2025-12-02T15:03:47Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+    }
+
+    /// Puts a value into the specified table.
+    pub fn put<T: Table>(&self, key: T::Key, value: T::Value) -> ProviderResult<()> {


looks like these can be accepted by ref?

mattsse · 2025-12-02T15:04:49Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+#[derive(Debug)]
+struct RocksDBProviderInner {
+    db: DB,
+    metrics: Option<Arc<RocksDBMetrics>>,


I assume we don't actually need the arc wrapper here?

mattsse · 2025-12-02T15:05:30Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+/// Converts Reth's [`LogLevel`] to `RocksDB`'s [`rocksdb::LogLevel`].
+const fn convert_log_level(level: LogLevel) -> rocksdb::LogLevel {
+    match level {
+        LogLevel::Fatal => rocksdb::LogLevel::Fatal,
+        LogLevel::Error => rocksdb::LogLevel::Error,
+        LogLevel::Warn => rocksdb::LogLevel::Warn,
+        LogLevel::Notice | LogLevel::Verbose => rocksdb::LogLevel::Info,
+        LogLevel::Debug | LogLevel::Trace | LogLevel::Extra => rocksdb::LogLevel::Debug,
+    }
+}


can we move this to end of file

Rjected · 2025-12-02T17:09:17Z

I recommend temporarily disabling RocksDB support on Windows and using libmdbx as the fallback. Improving Windows compatibility for RocksDB can be considered future work, wdyt ?

This makes sense to me

Rjected

makes sense to me but left some nits

Rjected · 2025-12-02T17:11:58Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+impl RocksDBBuilder {
+    /// Creates a new builder with optimized default options.
+    pub fn new(path: impl AsRef<Path>) -> Self {
+        let cache = Cache::new_lru_cache(128 << 20);


I would create a default cache size const and use it here

Rjected · 2025-12-02T17:13:53Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+        let mut table_options = rocksdb::BlockBasedOptions::default();
+        table_options.set_block_size(16 * 1024);
+        table_options.set_cache_index_and_filter_blocks(true);
+        table_options.set_pin_l0_filter_and_index_blocks_in_cache(true);
+        // Shared block cache for all column families.
+        table_options.set_block_cache(cache);
+        // Bloom filter: 10 bits/key = ~1% false positive rate, full filter for better read
+        // performance. this setting is good trade off a little bit of memory for better
+        // point lookup performance. see https://github.com/facebook/rocksdb/wiki/RocksDB-Bloom-Filter#configuration-basics
+        table_options.set_bloom_filter(10.0, false);
+        table_options.set_optimize_filters_for_memory(true);
+
+        let mut options = Options::default();
+        options.set_block_based_table_factory(&table_options);
+        options.create_if_missing(true);
+        options.create_missing_column_families(true);
+        options.set_max_background_jobs(6);
+        options.set_bytes_per_sync(1_048_576);
+
+        options.set_bottommost_compression_type(rocksdb::DBCompressionType::Zstd);
+        options.set_bottommost_zstd_max_train_bytes(0, true);
+        options.set_compression_type(rocksdb::DBCompressionType::Lz4);
+        options.set_compaction_pri(rocksdb::CompactionPri::MinOverlappingRatio);


nit: I'd import DBCompressionType, CompactionPri, BlockBasedOptions at the top of the file

Rjected · 2025-12-02T17:16:09Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+        // Follow recommend tuning guide from RocksDB wiki, see https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning
+        let mut table_options = rocksdb::BlockBasedOptions::default();
+        table_options.set_block_size(16 * 1024);
+        table_options.set_cache_index_and_filter_blocks(true);
+        table_options.set_pin_l0_filter_and_index_blocks_in_cache(true);
+        table_options.set_block_cache(cache);
+        // Bloom filter: 10 bits/key = ~1% false positive rate, full filter for better read
+        // performance. this setting is good trade off a little bit of memory for better
+        // point lookup performance. see https://github.com/facebook/rocksdb/wiki/RocksDB-Bloom-Filter#configuration-basics
+        table_options.set_bloom_filter(10.0, false);
+        // Optimize filters for memory.
+        table_options.set_optimize_filters_for_memory(true);


maybe we could have a fn default_table_options(cache: &Cache) -> BlockBasedOptions? So that we don't have to repeat this in both default_column_family_options and default_options.

Rjected · 2025-12-02T17:26:49Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+impl<'a> fmt::Debug for RocksDBBatch<'a> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        f.debug_struct("RocksDBBatch")
+            .field("provider", &self.provider)
+            .field("batch", &"<WriteBatch>")
+            .field("length", &self.inner.len())
+            .field("size_in_bytes", &self.inner.size_in_bytes())
+            .finish()


can we add some comments for what length and size_in_bytes represent w.r.t. the WriteBatch

duyquang6 · 2025-12-03T06:11:44Z

I recommend temporarily disabling RocksDB support on Windows and using libmdbx as the fallback. Improving Windows compatibility for RocksDB can be considered future work, wdyt ?

This makes sense to me

Hi sir @Rjected,
I’ve been able to compile on Windows, but the process is still quite manual and I haven’t validated runtime behavior yet. So I still think disable RocksDB support on windows is better at this time

For reference, here are the steps in case we decide to support this on Windows in the future:

Download the precompiled RocksDB library from MSYS2: https://packages.msys2.org/packages/mingw-w64-x86_64-rocksdb (Version 10.4.2 is compatible with rust-rocksdb 0.24)
Extract it, then build using:

ROCKSDB_LIB_DIR="{EXTRACTED_DIR}/mingw64/lib/" cargo build --target x86_64-pc-windows-gnu

duyquang6 · 2025-12-04T09:05:45Z

.github/workflows/lint.yml

        partition: [1, 2]
        total_partitions: [2]
-    timeout-minutes: 30
+    timeout-minutes: 60


issue: https://github.com/paradigmxyz/reth/actions/runs/19922208464/job/57115796165?pr=20071
I think because we add more dependency so this check is take longer

duyquang6 · 2025-12-04T09:38:20Z

The conditional compilation has been updated, and the changes are ready for review.

Rjected

just have one nit otherwise this looks good to me, think we can start implementing providers pretty easily after this

crates/storage/provider/src/providers/rocksdb/provider.rs

Co-authored-by: Dan Cline <[email protected]>

mattsse

this lgtm, with one pedantic suggestion

pending @Rjected

mattsse · 2025-12-05T11:35:44Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+
+/// Some types don't support compression (eg. B256), and we don't want to be copying them to the
+/// allocated buffer when we can just use their reference.
+macro_rules! compress_to_buf_or_ref {


this makes sense

mattsse · 2025-12-05T11:36:25Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+    /// Creates a new `RocksDB` provider.
+    pub fn new(path: impl AsRef<Path>) -> ProviderResult<Self> {
+        RocksDBBuilder::new(path).build()
+    }


would be nice to also have a fn builder convenience fn here, then you don't need to import two types

mattsse · 2025-12-05T11:38:01Z

crates/storage/provider/src/providers/rocksdb/provider.rs

+        table: &'static str,
+        f: impl FnOnce(&Self) -> T,
+    ) -> T {
+        let start = std::time::Instant::now();


this start is redundant if no metrics, so we can make this an optional as well and then simply do a match over both options for recording

Rjected

LGTM

duyquang6 requested review from gakonst, joshieDo, rakita and shekhirin as code owners December 2, 2025 10:40

github-project-automation bot added this to Reth Tracker Dec 2, 2025

github-project-automation bot moved this to Backlog in Reth Tracker Dec 2, 2025

duyquang6 force-pushed the push-noznwmlszymt branch 11 times, most recently from 0dabfd4 to 127daf4 Compare December 2, 2025 12:19

duyquang6 marked this pull request as draft December 2, 2025 12:28

feat: rocksdb provider

066005b

duyquang6 force-pushed the push-noznwmlszymt branch 2 times, most recently from 971e7e8 to c96f5c8 Compare December 2, 2025 12:51

fix: clippy

239301e

duyquang6 force-pushed the push-noznwmlszymt branch 2 times, most recently from 3787628 to 239301e Compare December 2, 2025 14:39

duyquang6 marked this pull request as ready for review December 2, 2025 15:00

mattsse reviewed Dec 2, 2025

View reviewed changes

Rjected reviewed Dec 2, 2025

View reviewed changes

duyquang6 marked this pull request as draft December 4, 2025 06:37

duyquang6 force-pushed the push-noznwmlszymt branch 6 times, most recently from a2bf641 to 0111d9c Compare December 4, 2025 07:34

duyquang6 marked this pull request as ready for review December 4, 2025 07:59

duyquang6 requested a review from DaniPopes as a code owner December 4, 2025 07:59

duyquang6 force-pushed the push-noznwmlszymt branch from 0111d9c to 412314a Compare December 4, 2025 08:16

duyquang6 added 2 commits December 4, 2025 15:17

chore: rocksdb compile on unix only

8ffebcb

fix: increase timeout for crate-checks

04cc199

duyquang6 force-pushed the push-noznwmlszymt branch from 412314a to 8ffebcb Compare December 4, 2025 08:17

duyquang6 commented Dec 4, 2025

View reviewed changes

Rjected reviewed Dec 4, 2025

View reviewed changes

crates/storage/provider/src/providers/rocksdb/provider.rs Outdated Show resolved Hide resolved

Update crates/storage/provider/src/providers/rocksdb/provider.rs

5033d3a

Co-authored-by: Dan Cline <[email protected]>

duyquang6 marked this pull request as draft December 5, 2025 02:07

fix lint

fbe1b12

duyquang6 force-pushed the push-noznwmlszymt branch from a09759c to fbe1b12 Compare December 5, 2025 02:17

duyquang6 marked this pull request as ready for review December 5, 2025 02:42

mattsse approved these changes Dec 5, 2025

View reviewed changes

github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Dec 5, 2025

fix: comments

9fbb6f2

duyquang6 force-pushed the push-noznwmlszymt branch from 11e8a60 to 9fbb6f2 Compare December 5, 2025 12:10

Rjected approved these changes Dec 5, 2025

View reviewed changes

Rjected added this pull request to the merge queue Dec 5, 2025

Merged via the queue into paradigmxyz:main with commit 00ccb2b Dec 5, 2025
44 checks passed

github-project-automation bot moved this from In Progress to Done in Reth Tracker Dec 5, 2025

feat(persistence): implement RocksDB provider #20071

feat(persistence): implement RocksDB provider #20071

Conversation

duyquang6 commented Dec 2, 2025

Uh oh!

duyquang6 commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattsse left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rjected commented Dec 2, 2025

Uh oh!

Rjected left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

duyquang6 commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

duyquang6 commented Dec 4, 2025

Uh oh!

Rjected left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattsse left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rjected left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

duyquang6 commented Dec 2, 2025 •

edited

Loading