[runtime/deterministic] Support realtime
#1848
Conversation
patrick-ogrady
changed the title
realtimerealtime
Deploying monorepo with
|
| Latest commit: |
58d717b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://bdf97f56.monorepo-eu0.pages.dev |
| Branch Preview URL: | https://runtime-real-wait.monorepo-eu0.pages.dev |
|
I suppose determinism in this case can depend wholly on when the application actually returns (and that the variance must be contained entirely within You can imagine a simple case where a task that returns when polled the ith time will have a different execution pattern when returned the jth time (again because we don't determine when the external process returns). |
|
The complete fix here is to have a special "poll" that resolves at some timeout (panicking if the task doesn't complete by that point). We can add jitter to the timeout with the seed to still add some interesting sequencing dynamics. |
|
Instead of panicking at the end of the Range, we should consider blocking forever (will be much less flaky -> we can warn)? |
Similar to how cargo test says that a test ran too long? Yeah, I think that makes sense. |
Yeah, exactly. This is probably the right balance of "hostility" from the API (without requiring users to set a long bound). It also makes it super easy to set a very wide range of execution duration to explore pretty odd execution paths. |
| *this.future = None; | ||
| break Poll::Ready(value); | ||
| } | ||
| Poll::Pending => blocker.wait(), |
There was a problem hiding this comment.
If this is the "first loop", emit a warning log.
There was a problem hiding this comment.
With the "Range" logic, it is common to underestimate the actual value to create more unique scenarios. We can probably defer this as a result.
There was a problem hiding this comment.
After thinking about this for a few more days, opted to not add logging in this case.
| /// | ||
| /// If we are operating in [Config::realtime], we will always try to keep making | ||
| /// progress after the passage of time by polling all pending tasks. | ||
| fn assert_liveness(&self) { |
There was a problem hiding this comment.
Now that we have the Pacer, it may be tempting to delete the actual sleep handling here but we are opting to keep it for now because external processes may receive some timeout invocation after context.sleep() (and if that moves virtually, it could end much sooner than it should).
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for "realtime" mode in the deterministic runtime to enable testing with external processes. It introduces a Pacer trait for controlling future execution timing and implements synchronization primitives to coordinate between deterministic and real-time execution.
- Adds a
realtimeconfiguration option that disables time simulation and stall detection - Implements the
Pacertrait with deterministic pacing using random delays and blocking - Introduces a
Blockersynchronization primitive for thread coordination via wakers
Reviewed Changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| runtime/src/utils/mod.rs | Adds Blocker synchronization primitive and comprehensive tests |
| runtime/src/utils/cell.rs | Implements Pacer trait delegation for Cell wrapper |
| runtime/src/tokio/runtime.rs | Implements passthrough Pacer for tokio runtime |
| runtime/src/lib.rs | Defines Pacer trait and FutureExt extension |
| runtime/src/deterministic.rs | Core realtime support with pacing implementation and time management |
| runtime/Cargo.toml | Adds pin-project-lite dependency |
| Cargo.toml | Adds pin-project-lite to workspace dependencies |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Before merging, I also plan to:
|
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## main #1848 +/- ##
==========================================
+ Coverage 92.20% 92.21% +0.01%
==========================================
Files 305 308 +3
Lines 79509 81571 +2062
==========================================
+ Hits 73310 75221 +1911
- Misses 6199 6350 +151
... and 22 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
3 participants
Fixes: #1838
To support testing with external processes (that aren't managed by
runtime), we must wait some "realtime" for said processes to perform useful work (without allowing tasks tracked by the deterministic runtime to "run ahead").