test: stop CI flakes from tight wall-clock deadlines#103
Conversation
…akes A class of unit tests bounds real retry/resolve/hydration work with a tight wall-clock deadline (100-1000 ms) and asserts a definite outcome (success, or a specific terminal code after a bounded retry). On slow, contended GitHub runners the work overran the budget and the test got a Timeout instead of its asserted outcome, flaking the Test job while every local run passed. Widen those budgets to the suite's 5 s idiom so the asserted state is reached deterministically. Genuine deadline-expiry tests (permanently-unresolvable adapters, single-shot timeout_ms 0, deadline-value plumbing, lock-contention) keep their tight budgets - a slow runner still expires them correctly. Test-only; no production change.
Greptile SummaryThis PR widens wall-clock test deadlines from 100–1,000 ms to 5,000 ms across 13 sites in 12 test files to eliminate CI flakes caused by slow GitHub macOS runners. No production code is touched.
Confidence Score: 5/5Safe to merge — purely test-only numeric constant changes with no production impact. Every change is a one-for-one numeric swap in a test file. The root cause of the flakes is clearly traced: TerminalAfterRetryAdapter::get_element_bounds sleeps 120 ms before returning its stale_ref error, which the old 100 ms deadline would expire before the sleep completed, substituting TIMEOUT for the asserted STALE_REF. Widening to 5,000 ms lets the sleep finish and the correct error propagate. The same pattern applies across all 13 sites. Genuine expiry tests (timeout_ms: 0, permanently-failing adapters) are correctly left untouched since their outcomes are deadline-independent. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Test sets deadline
100–1000 ms] -->|Slow CI runner| B{Work completes
before deadline?}
B -->|No — budget overrun| C[TIMEOUT error
❌ Flake]
B -->|Yes — local/fast runner| D[Expected error or value
✅ Pass]
E[Test sets deadline
5000 ms] -->|Slow CI runner| F{Work completes
before deadline?}
F -->|Yes — deterministic| G[Expected error or value
✅ Pass]
H[Genuine expiry test
timeout_ms 0 / permanent adapter] --> I{Budget expires?}
I -->|Always, regardless of speed| J[TIMEOUT / expiry
✅ Unchanged — left alone]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[Test sets deadline
100–1000 ms] -->|Slow CI runner| B{Work completes
before deadline?}
B -->|No — budget overrun| C[TIMEOUT error
❌ Flake]
B -->|Yes — local/fast runner| D[Expected error or value
✅ Pass]
E[Test sets deadline
5000 ms] -->|Slow CI runner| F{Work completes
before deadline?}
F -->|Yes — deterministic| G[Expected error or value
✅ Pass]
H[Genuine expiry test
timeout_ms 0 / permanent adapter] --> I{Budget expires?}
I -->|Always, regardless of speed| J[TIMEOUT / expiry
✅ Unchanged — left alone]
Reviews (1): Last reviewed commit: "test: widen wall-clock budgets on retry/..." | Re-trigger Greptile |
Summary
A class of unit tests bounds real retry/resolve/hydration work with a tight wall-clock deadline (100–1000 ms) and asserts a definite outcome. On slow, contended GitHub macOS runners the work overran the budget and the test got a
Timeoutinstead of its asserted outcome, flaking theTestjob while every local run passed. Three had already surfaced on CI (selected_hydration,drag_retries...occlusion,terminal_stale_ref...).This widens those budgets to the suite's existing 5 s idiom so the asserted state is reached deterministically. 13 sites across 12 test files; test-only, no production change.
Genuine deadline-expiry tests keep their tight budgets — permanently-unresolvable adapters, single-shot
timeout_ms 0, deadline-value plumbing, and lock-contention tests all expire correctly regardless of runner speed, so widening them would be wrong (or slow) and they were left untouched.Verification
cargo test --lib -p agent-desktop-core— 978 pass, 0 failcargo test --lib --workspace— all suites passcargo fmt --all -- --check— clean