Queue 105: make ExecutionMonitor tests deterministic - #287
Conversation
…:sleep
test_execution_monitor_successful_execution_with_progress slept 10 ms per
iteration and then asserted a 50 ms no-progress budget had not been exceeded.
thread::sleep guarantees a minimum duration, never a maximum, so that assertion
is not something the standard library promises -- on a contended runner the
sleep overruns and the test fails. It went red on GitHub's hosted runners, and a
docs-only branch off main reproduced it with every .rs file byte-identical,
which is what confirmed it had nothing to do with the code under test.
The timeout logic is already a pure function of the instants it is handed; only
the source of those instants was hard-wired. Add new_at / check_at /
mark_progress_at taking an explicit Instant, with new / check / mark_progress
delegating to them with Instant::now(). They are pub(crate), so the public API
is unchanged.
The five tests become nine, all deterministic and none sleeping:
- no-progress and wall-clock timeouts still fire
- boundary behaviour at exactly the limit and one millisecond past it, in both
directions -- untestable with sleeps, and previously uncovered
- mark_progress restarts the no-progress window
- mark_progress does NOT extend the absolute wall-clock budget
- steady work with regular progress marks survives 200 iterations over 8 s of
simulated time, well past the 50 ms no-progress budget (the property the
flaky test was reaching for, now actually exercised)
- max_wall_clock_ratio and max_wall_clock_s combine so the tighter wins, in
both orders
Suite time for this group goes from ~140 ms of sleeping to 0.00 s, and 10 runs
under 2x CPU oversubscription pass identically.
Closes #284
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
ExecutionMonitor::check_at can panic on non-monotonic instants due to duration_since, which should be guarded now that the clock is injectable.
Pull request overview
This PR makes ExecutionMonitor’s timeout logic testable without relying on real wall-clock sleeps by introducing pub(crate) entrypoints that accept an explicit Instant, enabling deterministic, boundary-precise unit tests while keeping the public API unchanged.
Changes:
- Add
ExecutionMonitor::{new_at, check_at, mark_progress_at}with existingnew/check/mark_progressdelegating toInstant::now(). - Replace sleep-based
ExecutionMonitorunit tests with deterministic tests that drive time via fixed instants, adding boundary coverage.
File summaries
| File | Description |
|---|---|
| core/src/sim.rs | Adds injectable-clock variants of ExecutionMonitor methods and rewrites/expands unit tests to be deterministic and cover boundaries. |
Review details
Suppressed comments (1)
core/src/sim.rs:2747
check_atnow accepts an arbitraryInstant, but it still usesInstant::duration_since, which panics ifnowis earlier thanstart_timeorlast_progress. Since this is a new injectable-clock API (even ifpub(crate)), it’s safer to avoid introducing a panic footgun; usechecked_duration_sinceand return an error on non-monotonic instants.
if let Some(max_wall_clock) = self.max_wall_clock
&& now.duration_since(self.start_time) > max_wall_clock
{
bail!(
"Execution timeout ({context}): exceeded wall-clock limit of {:.2} s",
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
CI: this PR and #285 fix exactly what the other still fails onNeither PR is green alone, because each is missing the other's fix. Together they are. This PR (#287), branched off
Both are fixed in #285 (golden thresholds of 1905.00 m replaced with physical bounds). #285, conversely, has every ESKF and RBPF test passing and fails on exactly one thing: So the merge order does not matter; whichever lands second turns CI green. One unrelated failure, newly surfacedmacOS also shows: This PR cannot reach the EKF. |
v1/p-exec-monitor-clockmainWhy
test_execution_monitor_successful_execution_with_progressslept 10 ms per iteration and thenasserted a 50 ms no-progress budget had not been exceeded.
thread::sleepguarantees a minimumduration, never a maximum, so that assertion is not something the standard library promises. It went
red on GitHub's hosted runners.
It is not caused by any code change: a docs-only branch off
main(#279) reproduces it withevery
.rsfile byte-identical. It was also the only remaining failure on #285 after the ESKFfixes landed, so it is currently the last thing between this repo and green CI.
What changed
The timeout logic is already a pure function of the instants it is handed; only the source of those
instants was hard-wired to
Instant::now(). Addednew_at/check_at/mark_progress_attakingan explicit
Instant, with the existing methods delegating. They arepub(crate)— the public APIis unchanged.
Five tests become nine, all deterministic, none sleeping:
mark_progressrestarts the no-progress windowmark_progressdoes not extend the absolute wall-clock budgetmax_wall_clock_ratioandmax_wall_clock_scombine so the tighter wins, in both ordersVerification
thread::sleepremains anywhere incore/src/sim.rsCloses #284
Branches off
main; independent of the spine.Queue:
docs/V1_QUEUE.md| Board: https://github.com/users/jbrodovsky/projects/7