From de4f97f43688e76d991864fd8681cc8f07f3eda5 Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Thu, 6 Aug 2026 21:21:31 +0200 Subject: [PATCH 1/3] fix(ci): a Verify-Filter matching zero artifacts must not pass the gate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The verification gate is a REQUIRED check, and it could be silenced by a one-word change to the PR body. `scripts/run-falcon-verification.py` exits 0 when its filter matches nothing — correctly, in isolation: no artifact ran, so none failed. But the gate's verdict is exactly that exit code (`steps.verify.outcome`), so a `Verify-Filter:` that matches zero artifacts turns a required safety gate green while verifying nothing. It even renders "✅ Rivet verification gate — 0/0 passed" into the sticky PR comment, so the deception is not visible to a reviewer skimming it. The PR body is attacker-controllable. The INJECTION risk was already handled (env-bound, quoted, never interpolated into run:) — but the SEMANTIC trust of the filter was not: an untrusted string decides how much verification happens, and "none" was an accepted answer. Caught empirically, not by review: `(has-tag "traceability")` reads perfectly plausible and matches 0 of the 195 sw-verification artifacts, because that tag lives on sw-reqs, not on verifications. I had put exactly that filter on #341. Fix: after the sweep, fail if the script reports `0 artifact(s) matched`. Verified locally: (has-tag "traceability") -> 0 artifacts -> guard fires (has-tag "oci") -> 1 artifact -> no false positive (has-tag "falcon") -> 156 artifacts (the default) Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HvusAXYbHLyv3uTzfBcMbG --- .github/workflows/verification-gate.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/.github/workflows/verification-gate.yml b/.github/workflows/verification-gate.yml index 47fa8bc..bd6c4ed 100644 --- a/.github/workflows/verification-gate.yml +++ b/.github/workflows/verification-gate.yml @@ -199,6 +199,26 @@ jobs: --filter "$FILTER" \ --markdown | tee verification-output.md + # A filter that matches NOTHING must not pass this gate. + # + # The script exits 0 on an empty match set — nothing ran, so nothing + # failed — which means a typo'd (or hostile) `Verify-Filter:` in the + # PR body silently green-lights a REQUIRED check while verifying + # zero artifacts. The body is attacker-controllable, so this is a + # gate-silencing vector, not just an ergonomic foot-gun: the + # injection risk was already handled above (env-bound, quoted), but + # the SEMANTIC trust of the filter was not. + # + # Caught empirically: `(has-tag "traceability")` reads perfectly + # plausible and matches 0 of the 195 sw-verification artifacts + # (the tag lives on sw-reqs, not verifications) — exit code 0. + if grep -qE '^# 0 artifact\(s\) matched' verification-output.md; then + echo "::error::Verify-Filter matched 0 verification artifacts — \ +refusing to pass a gate that verified nothing. Fix the filter or drop the \ +override to use the default." + exit 1 + fi + - name: Upload verification output if: always() && steps.scope.outputs.run_full == 'true' uses: actions/upload-artifact@v7 From 059bac10dd78f7f59d0934529476de6fd9a35192 Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Sat, 8 Aug 2026 10:02:44 +0200 Subject: [PATCH 2/3] fix(ci): raise the verification-gate timeout 90 -> 150 (STOPGAP, see #350) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The sweep has grown into its own ceiling. Measured across one day, same 156-artifact sweep: #340 ~63m success #344 1h30m21s FAILURE — killed at timeout-minutes: 90 #344 (earlier) cancelled #342 failed twice the same way #343 success twice Variance now exceeds headroom, so a REQUIRED check is decided by runner load rather than by whether anything is wrong. And a timeout surfaces as `fail`, indistinguishable from a real failure unless you read the duration — which already cost a diagnosis cycle. Worst of all, it was blocking the two PRs meant to improve this gate (#342 fail-open fix, #343 sweep speedup). Raising the ceiling does not weaken the gate: it is being KILLED, not failing. Landing it here because this PR already owns this file and is itself blocked by the timeout it fixes. THIS IS A STOPGAP and the third reactive bump (60 -> 90 -> 150). The trend is the real problem: 45m in July, >=90m now. #350 tracks it, and names the next measurement — the compile-vs-test split ON the runner. The sweep spans 64 crates with 29 `--release` steps, so compilation is the likely dominant cost; whole-crate -> named conversion (#343/#262) is right for traceability but, measured, will not close a 30-minute gap on its own (the non-falcon-core offenders run in 0-5s). Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HvusAXYbHLyv3uTzfBcMbG --- .github/workflows/verification-gate.yml | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/.github/workflows/verification-gate.yml b/.github/workflows/verification-gate.yml index bd6c4ed..9ad87ff 100644 --- a/.github/workflows/verification-gate.yml +++ b/.github/workflows/verification-gate.yml @@ -51,7 +51,24 @@ jobs: # 90 (was 60): a legit full sweep measured 45m21s (2026-07-11) but a # loaded self-hosted runner tipped it to 61m and FALSE-CANCELLED green # Dependabot PRs (2026-07-21). Headroom, not a slower gate. - timeout-minutes: 90 + # + # 150 (was 90, 2026-08-08): the sweep has grown into the ceiling AGAIN. + # Measured across one day: ~63m (#340, passed) up to >=90m (#344, KILLED at + # 1h30m21s), with #342 failing twice the same way. Variance now exceeds + # headroom, so a REQUIRED check was being decided by runner load rather + # than by whether anything is wrong. It also blocked the two PRs meant to + # improve this very gate. + # + # THIS IS A STOPGAP, and the third reactive bump in a row (60 -> 90 -> 150). + # The TREND is the problem: 45m in July, >=90m now. Raising the ceiling buys + # time; it does not address why the sweep grew, and shipping it alone just + # moves the cliff. Tracked in #350, where the next step is measuring the + # COMPILE-vs-TEST split on the runner: the sweep spans 64 crates with 29 + # `--release` steps (two profile-wide compile sets), so compilation is the + # likely dominant cost. Note that converting whole-crate steps to named ones + # (#343 / #262) demonstrably will NOT close a 30-minute gap on its own — + # measured, the non-falcon-core offenders run in 0-5s. + timeout-minutes: 150 env: CARGO_TERM_COLOR: always CARGO_INCREMENTAL: 0 From 75e74f554c9e73b3e709a554d15b49b2baf8defc Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Sat, 8 Aug 2026 10:03:40 +0200 Subject: [PATCH 3/3] =?UTF-8?q?fix(ci):=20my=20zero-match=20guard=20emitte?= =?UTF-8?q?d=20INVALID=20YAML=20=E2=80=94=20backslash=20continuation=20at?= =?UTF-8?q?=20column=200?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Self-inflicted, and it broke the very gate the guard was meant to strengthen. The guard's error message used a backslash-continued string whose continuation lines started at COLUMN 0: echo "::error::Verify-Filter matched 0 ... — \ refusing to pass a gate that verified nothing. Fix the filter or drop the \ override to use the default." Inside a `run: |` block scalar, a line at column 0 TERMINATES the block. The whole workflow file was therefore invalid YAML from the moment I added the guard, which is almost certainly why this PR's verification gate failed twice — NOT the 90-minute timeout I attributed it to in the previous commit. `python3 -c "yaml.safe_load(...)"` reports: ScannerError: while scanning a simple key, line 234 ... could not find expected ':' Fixed by keeping the message on one line, with a comment saying why, so the next person does not "tidy" it back into a continuation. Lesson worth the comment: I verified this file for the thing I was thinking about (no `${{ }}` inside `run:` — which I DID check, with a parser) and not for whether it still parsed at all. Both checks now run. YAML parses: OK, timeout-minutes=150 ${{ }} inside run: blocks: none Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HvusAXYbHLyv3uTzfBcMbG --- .github/workflows/verification-gate.yml | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/.github/workflows/verification-gate.yml b/.github/workflows/verification-gate.yml index 9ad87ff..b7f8964 100644 --- a/.github/workflows/verification-gate.yml +++ b/.github/workflows/verification-gate.yml @@ -230,9 +230,11 @@ jobs: # plausible and matches 0 of the 195 sw-verification artifacts # (the tag lives on sw-reqs, not verifications) — exit code 0. if grep -qE '^# 0 artifact\(s\) matched' verification-output.md; then - echo "::error::Verify-Filter matched 0 verification artifacts — \ -refusing to pass a gate that verified nothing. Fix the filter or drop the \ -override to use the default." + # NB: keep this on ONE line. A backslash-continued string whose + # continuation starts at column 0 terminates the `run: |` block + # scalar and makes the whole workflow file INVALID YAML — which is + # how the first version of this guard broke its own gate. + echo "::error::Verify-Filter matched 0 verification artifacts — refusing to pass a gate that verified nothing. Fix the filter, or drop the override to use the default." exit 1 fi