Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more metrics for compaction and sealed replicas and enable compaction after bootstrap #3006

Merged

Conversation

justinlin-linkedin
Copy link
Collaborator

Summary

When a replica starts bootstrapping on a host, we disable compaction for this replica. This makes sense since bootstrap process employs the regular replication protocol, which would ignore expired and deleted blob. There are not many blobs to compact. However, after bootstrap finishes, this replica would start serving client requests, more blobs will be deleted and expired. We should enable compaction after bootstrap.

This PR fixed this issue by adding logic to enable compaction in storage manager.

This PR also exposes two more metrics

  1. showing how many replicas are disabled for compaction on each host
  2. showing how many replicas are sealed on each host.

Testing Done

Wait for unit test

@codecov-commenter
Copy link

codecov-commenter commented Feb 14, 2025

Codecov Report

Attention: Patch coverage is 46.29630% with 29 lines in your changes missing coverage. Please review.

Project coverage is 69.93%. Comparing base (52ba813) to head (b05c094).
Report is 179 commits behind head on master.

Files with missing lines Patch % Lines
...in/java/com/github/ambry/store/StorageManager.java 23.80% 11 Missing and 5 partials ⚠️
...m/github/ambry/clustermap/HelixClusterManager.java 0.00% 9 Missing ⚠️
.../main/java/com/github/ambry/store/DiskManager.java 66.66% 1 Missing and 1 partial ⚠️
...java/com/github/ambry/store/CompactionManager.java 50.00% 0 Missing and 1 partial ⚠️
.../com/github/ambry/store/StorageManagerMetrics.java 85.71% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3006      +/-   ##
============================================
+ Coverage     64.24%   69.93%   +5.68%     
- Complexity    10398    12159    +1761     
============================================
  Files           840      889      +49     
  Lines         71755    75202    +3447     
  Branches       8611     9000     +389     
============================================
+ Hits          46099    52590    +6491     
+ Misses        23004    19859    -3145     
- Partials       2652     2753     +101     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@Arun-LinkedIn Arun-LinkedIn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@justinlin-linkedin justinlin-linkedin merged commit 87d2a19 into linkedin:master Feb 20, 2025
5 checks passed
@justinlin-linkedin justinlin-linkedin deleted the justin/compaction branch February 20, 2025 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants