db_bench support user defined timestamp

[jowlyzhang]

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev

Expected behavior

db_bench support enabling user defined timestamp

Actual behavior

some errors encountered in while benchmarking this feature with the tool

Steps to reproduce the behavior

Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://groups.google.com/forum/#!forum/rocksdb or https://www.facebook.com/groups/rocksdb.dev
./db_bench --num=10000000 --disable_wal --value_size=16 --report_interval_seconds=1 --db=/tmp/timestamp --statistics --benchmarks=fillrandom --delayed_write_rate=$((12810241024)) --max_background_flushes=4 --max_background_compactions=8 --threads=16 -user_timestamp_size=8 --compression_type=none

./db_bench --num=10000000 --disable_wal --value_size=16 --report_interval_seconds=1 --db=/tmp/timestamp --statistics --benchmarks=readrandom --delayed_write_rate=$((12810241024)) --cache_size=$(( 4 *1024 * 1024)) --max_write_buffer_number=6 --max_background_flushes=4 --max_background_compactions=8 -user_timestamp_size=8 --threads=1 --compression_type=none --use_existing_db --duration=60 --read_with_latest_user_timestamp

On the write path: the keys that are created are without a timestamp. The command batch.clear() also reset the value of default_cf_ts_sz_ and the db_bench does not set it back
On the read path: only a small fraction of the keys, this is suspected to be an issue with the tool instead of the feature's own bug

[dorlevi]

@jowlyzhang : thanks for assigning this to yourself. Do you know when you could pick this issue up?

[jowlyzhang]

@jowlyzhang : thanks for assigning this to yourself. Do you know when you could pick this issue up?

I don't think I will be able to check this soon, are you interested in taking it?

[dorlevi]

We are blocked on this issue -- but don't think we have the capacity to take this ourselves as we aren't as familiar with this codebase/path

[jowlyzhang]

We are blocked on this issue -- but don't think we have the capacity to take this ourselves as we aren't as familiar with this codebase/path

Thank you for this context. I will check this out soon by end of next week.

[dorlevi]

hi, I wanted to check back in, thank you

[hilikspdb]

LDB scan and get do not work as well. The issue in LDB is that the caller must give timestamp for new iterator & Get (unlike snapshot where no snapshot means the latest). LDB can not create a timestamp without a knowledge on the cooperator. I think that in order to make ldb works the simplest way is to support null timestamp (get the latest version in get ; get all the versions in scan) in the read options.

[dorlevi]

@jowlyzhang -- wanted to check in again and see if this is planned to be addressed

[jowlyzhang]

I was actually recently working on this, the ./ldb get type of commands, like multiget, scan, get_entity etc should have read timestamp support after this PR #12641

I'm currently working on dump, wal_dump and idump for similar support and I will later work on the db_bench one.

[dorlevi]

Amazing ! Thank you so much we will stay tuned and test it out after

…

On Wed, May 22 2024 at 1:48 PM, Yu Zhang < ***@***.*** > wrote: I was actually recently working on this, the./ldb get type of commands, like multiget , scan , get_entity etc should have read timestamp support after this PR #12641 ( #12641 ) I'm currently working on dump , wal_dump and idump for similar support and I will later work on the db_bench one. — Reply to this email directly, view it on GitHub ( #12233 (comment) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AALD6HTSQ453VDKSG3IHIBLZDTLAPAVCNFSM6AAAAABBYYUYZCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRVGQYTOMJWGE ). You are receiving this because you are subscribed to this thread. Message ID: <facebook/rocksdb/issues/12233/2125417161 @ github. com>