Add write amp rate limiter #368
Conversation
|
/run-all-test |
|
/run-all-tests |
3 similar comments
|
/run-all-tests |
|
/run-all-tests |
|
/run-all-tests |
v01dstar
force-pushed
the
write-amp-rate-limiter
branch
from
8a4ebc9
to
afb4fc9
Compare
ti-chi-bot
bot
added
dco-signoff: yes
ti-chi-bot
bot
added
the
do-not-merge/work-in-progress
v01dstar
force-pushed
the
write-amp-rate-limiter
branch
from
afb4fc9
to
68c5156
Compare
ti-chi-bot
bot
removed
the
do-not-merge/work-in-progress
v01dstar
force-pushed
the
write-amp-rate-limiter
branch
from
68c5156
to
5e674d8
Compare
Signed-off-by: v01dstar <[email protected]> Signed-off-by: Yang Zhang <[email protected]>
v01dstar
force-pushed
the
write-amp-rate-limiter
branch
from
5e674d8
to
ca2cf28
Compare
|
@LykxSassinator: adding LGTM is restricted to approvers and reviewers in OWNERS files. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: Yang Zhang <[email protected]>
|
@SpadeA-Tang: adding LGTM is restricted to approvers and reviewers in OWNERS files. In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
wait for merging of #376. |
|
/lgtm refresh |
|
@wuhuizuo: adding LGTM is restricted to approvers and reviewers in OWNERS files. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: LykxSassinator, SpadeA-Tang, wuhuizuo The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
|
New changes are detected. LGTM label has been removed. |
|
Merge #381 with this for future cherry-pick. |
This PR are following PRs combined:
The WriteAmpBasedRateLimiter automatically adjusts rate limiting by predicting I/O trends using heuristic methods based on the write amplification factor. e.g. a high write throughput with
HIGHpriority (flush) implies, highly likely, there will beLOWpriority write throughput soon, because RocksDB needs to compact the L0 files to lower levels, it is better to provision in advance. Compared to the default rate limiter, it reduces the likelihood of triggering write stalls and helps prevent I/O jitter.Comments:
* Env::IOPriority has more options now,
LOW,MID,HIGH,USERinstead of justLOWandHIGH, write amplification rate limiter's tuning logic may need to adjust to this change.