Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 5.4] compaction: improve partition estimates when using garbage collected sstables #18626

Closed
wants to merge 1 commit into from

Conversation

mergify[bot]
Copy link

@mergify mergify bot commented May 11, 2024

When a compaction strategy uses garbage collected sstables to track expired tombstones, do not use complete partition estimates for both the regular output sstables and the temporary sstables. Instead split the partition estimate between the two based on the droppable tombstone ratio estimate.

Fixes #18283

(cherry picked from commit 36b98d8)

Refs #18465

When a compaction strategy uses garbage collected sstables to track
expired tombstones, do not use complete partition estimates for them,
instead, use a fraction of it based on the droppable tombstone ratio
estimate.

Fixes #18283

Signed-off-by: Lakshmi Narayanan Sreethar <[email protected]>
(cherry picked from commit 36b98d8)
@mergify mergify bot requested a review from nyh as a code owner May 11, 2024 21:46
@scylladb-promoter
Copy link
Contributor

🔴 CI State: FAILURE

❌ - Build

Build Failure:

[2024-05-11T22:02:55.601Z] FAILED: build/debug/compaction/compaction.o 
[2024-05-11T22:02:55.601Z] clang++ -MD -MT build/debug/compaction/compaction.o -MF build/debug/compaction/compaction.o.d -I/jenkins/workspace/scylla-5.4/scylla-ci/scylla/seastar/include -I/jenkins/workspace/scylla-5.4/scylla-ci/scylla/build/debug/seastar/gen/include -U_FORTIFY_SOURCE -Werror=unused-result -fstack-clash-protection -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -DSEASTAR_API_LEVEL=7 -DSEASTAR_BUILD_SHARED_LIBS -DSEASTAR_SSTRING -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_DEBUG -DSEASTAR_DEFAULT_ALLOCATOR -DSEASTAR_SHUFFLE_TASK_QUEUE -DSEASTAR_DEBUG_SHARED_PTR -DSEASTAR_LOGGER_TYPE_STDOUT -DSEASTAR_TYPE_ERASE_MORE -DBOOST_NO_CXX98_FUNCTION_BASE -DFMT_SHARED -I/usr/include/p11-kit-1   -ffile-prefix-map=/jenkins/workspace/scylla-5.4/scylla-ci/scylla=. -march=westmere -DDEBUG -DSANITIZE -DDEBUG_LSA_SANITIZER -DSCYLLA_ENABLE_ERROR_INJECTION -Og -DSCYLLA_BUILD_MODE=debug -g -gz -iquote. -iquote build/debug/gen --std=gnu++20  -ffile-prefix-map=/jenkins/workspace/scylla-5.4/scylla-ci/scylla=. -march=westmere  -DBOOST_TEST_DYN_LINK   -DNOMINMAX -DNOMINMAX -fvisibility=hidden -Wall -Werror -Wimplicit-fallthrough -Wno-mismatched-tags -Wno-c++11-narrowing -Wno-overloaded-virtual -Wno-unused-command-line-argument -Wno-unsupported-friend -Wno-implicit-int-float-conversion -Wno-psabi -Wno-narrowing -Wno-error=deprecated-declarations -DXXH_PRIVATE_API -DSEASTAR_TESTING_MAIN -DFMT_DEPRECATED_OSTREAM  -c -o build/debug/compaction/compaction.o compaction/compaction.cc
[2024-05-11T22:02:55.602Z] compaction/compaction.cc:739:116: error: too many arguments to function call, expected single argument gc_before, have 3 arguments
[2024-05-11T22:02:55.602Z]             sum_of_estimated_droppable_tombstone_ratio += sst->estimate_droppable_tombstone_ratio(gc_clock::now(), _table_s.get_tombstone_gc_state(), _schema);
[2024-05-11T22:02:55.602Z]                                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2024-05-11T22:02:55.602Z] ./sstables/sstables.hh:894:12: note: estimate_droppable_tombstone_ratio declared here
[2024-05-11T22:02:55.602Z]     double estimate_droppable_tombstone_ratio(gc_clock::time_point gc_before) const;
[2024-05-11T22:02:55.602Z]            ^
[2024-05-11T22:02:55.602Z] 1 error generated.

Build Details:

  • Duration: 20 min
  • Builder: i-0b684f058ba2570bf (m5d.8xlarge)

@mykaul
Copy link
Contributor

mykaul commented May 12, 2024

@lkshminarayanan - needs some manual work

@lkshminarayanan
Copy link
Member

Replaced by #18656

@mergify mergify bot deleted the mergify/copy/branch-5.4/pr-18465 branch May 13, 2024 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants