ci: add flaky-test management (#489) - #1
Merged
Conversation
Switch CI from `cargo test` to `cargo llvm-cov nextest` so a flaky test auto-retries (--retries 2) and surfaces as flaky-pass instead of hard-failing the PR, while coverage stays gated at >=80% lines through the same combined command (never split nextest-run and llvm-cov into separate steps - that collects zero coverage). Also drop the redundant full test-suite re-run from publish.yml (tag-triggered crates.io publish): the PR gates in ci.yml already tested this exact tree (branch protection requires up-to-date-with-main), so re-running it at release time only re-tests a green tree and was the #488 release-blocker (a flake post-merge fails publish with no PR-gate retry path). Deploy now does build + package + publish only. Co-Authored-By: Claude <noreply@anthropic.com>
… args (#489) cargo llvm-cov nextest treats args after `--` as test-binary args, which nextest rejects (--test-threads/--retries are nextest run options, not supported by the test binary), failing every CI run with exit 96. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cargo test->cargo llvm-cov nextest --all-features --workspace --fail-under-lines 80 -- --test-threads=1 --retries 2(single combined command, coverage preserved + gated >=80%, nextest auto-retries a flaky test up to 2x before failing).cargo-nextestinstall step toci.yml.testjob frompublish.yml(tag-triggered crates.io publish). PR gates inci.ymlalready tested this exact tree (branch protection requires up-to-date-with-main). Re-running it at release time only re-tests green and was the #488 release-blocker (post-merge flake fails publish with no PR-level retry).publish.ymlnow does build + package + publish only.Cargo.toml0.1.0 -> 0.1.1 (patch — CI-only change, no public API/behavior change).Stack: Rust (Cargo workspace, no package.json). release.yml (changelog+tag) already had no test job — untouched.
Test plan
ci.yml+publish.ymlYAML on this PR (workflow syntax checked by GH on push).#489
Co-Authored-By: Claude noreply@anthropic.com