From 20821194aeccfbc601c8d6685e8b48ed792b3a51 Mon Sep 17 00:00:00 2001 From: sepehr-safari Date: Thu, 30 Jul 2026 10:04:34 +0300 Subject: [PATCH] docs: require a scenario for every detection rule Three of the 16 detection rules shipped without a scenario exercising them: TIMEOUT_NO_HEARTBEAT, FIRMWARE_UPDATE_FAILURE and REPEATED_BOOT_NOTIFICATION. REPEATED_BOOT_NOTIFICATION shows how the gap opens: it landed in #114 and no scenario followed, because nothing in the guide asked for one. States the invariant at the top of "Contributing Detection Rules" and adds it as step 7 of "How to Add a Detection Rule", next to the existing false-positive audit, so both directions of coverage are requested in the same place. Step 6 confirms a rule stays quiet where it should; step 7 confirms it fires where it should. Closes #140 --- CONTRIBUTING.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e884071..5b2b8d6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -261,6 +261,13 @@ trace with expected failure outcomes and optional assertions. Detection rules identify failure patterns in traces. +**Every detection rule ships with a scenario that exercises it, added in the same +pull request as the rule.** A rule without a scenario has no regression guard, so +nothing in the corpus catches it breaking. This is the invariant that keeps the +scenario corpus honest as rules are added, and it means the scenario count in the +READMEs tracks real coverage rather than volume. See +[How to Add a Scenario](#how-to-add-a-scenario). + ### How to Add a Detection Rule 1. Add the failure code to `FailureCode` in `packages/toolkit/src/core/types.ts`. @@ -290,6 +297,11 @@ Detection rules identify failure patterns in traces. 6. **Audit all existing scenarios** — new rules may trigger on existing fixtures. Fix false positives or add the new code to `expectedFailures`. +7. **Add a scenario that exercises the new rule**, in this same pull request, + following [How to Add a Scenario](#how-to-add-a-scenario). This is the positive side of + step 6: step 6 confirms the rule stays quiet where it should, and this confirms + it fires where it should. A rule shipped without one is incomplete. + ## Contributing Good-First-Issues Good-first-issues are labeled with `good-first-issue` and are designed for