feat: fail closed on unverified head snapshot in dependency-vuln-check#749
Conversation
The base side of the dependency diff is resolved to a verified snapshotted ancestor and fails closed when it cannot be established. The head side had no equivalent guard: check.py trusted compare(effective_base...head_sha) and only failed closed if the API call itself errored — never verifying the head SBOM was actually submitted and indexed. If the head snapshot submission failed, or had not finished indexing when the compare ran, the head side of the diff read empty and a genuinely new vulnerable dependency passed silently (false negative). Harmless while the consuming gate was observe-only; under blocking it is a false green. Two guards close it: - Submission failure — new optional `head-snapshot-succeeded` input. The consumer passes the result of its head-SBOM submission job; when 'false' the gate fails closed (honors the override label, like the base-side paths). Unset preserves current behavior for existing consumers. - Indexing lag — when the head side of the diff looks empty (0 added, deps removed) the compare is re-fetched a few times with backoff to let the dependency graph settle. A removal-only PR has the same shape and simply re-confirms; the gate never blocks on this signal, it only waits out lag, so it cannot misfire into a false positive. 68/68 pytest pass (8 new: helper, gap-1 fail-closed + override bypass, gap-2 retry-then-block and persistent-empty-never-blocks). README + action inputs documented.
There was a problem hiding this comment.
Pull request overview
This PR strengthens dependency-vuln-check by adding head-side verification so the Dependency Review compare isn’t trusted when the PR head snapshot is missing/failed or still indexing, preventing false-negative “green” results when the gate becomes blocking.
Changes:
- Adds optional
head-snapshot-succeededaction input and a fail-closed path when it is explicitly'false'. - Adds a head-diff “looks empty” heuristic with retry + backoff to absorb dependency-graph indexing lag before evaluating the final diff.
- Adds unit tests for the new helper and head-guard behaviors, and updates README documentation.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| dependency-vuln-check/check.py | Adds head snapshot guard, empty-head retry logic, and the _looks_like_missing_head helper. |
| dependency-vuln-check/action.yml | Introduces head-snapshot-succeeded input and wires it into HEAD_SNAPSHOT_SUCCEEDED. |
| dependency-vuln-check/README.md | Documents the new input and head-side verification/lag behavior. |
| dependency-vuln-check/test_check.py | Adds tests covering submission-failed fail-closed behavior and indexing-lag retry behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot review on #749: the guard blocked only on the literal 'false', so a consumer wiring `needs.<job>.result` directly (yielding 'failure' / 'cancelled' / 'skipped') or a typo would silently be treated as "proceed" — turning the guard into a no-op and reintroducing the false negative. Now only '', 'true', 'success' proceed; every other value fails closed (the submission result is named in the reason). A wiring mistake fails safe (blocks, visible, override-able) instead of silently disabling the guard. README + action input doc updated; 76/76 pytest (added unexpected-value + recognized-ok parametrized tests).
| # not wire it) proceed; ANY other value — 'false', a raw job result like | ||
| # 'failure'/'cancelled'/'skipped', or a typo — fails closed, so a wiring | ||
| # mistake cannot silently turn this guard into a no-op. | ||
| if head_snapshot_ok not in ("", "true", "success"): |
There was a problem hiding this comment.
❓ The allow-list fix closes the failure / cancelled path nicely. One case it doesn't cover: success is still a "proceed" value, and the consumer's pr-maven-snapshot job is continue-on-error: true. If that masks a failed snapshot to needs.pr-maven-snapshot.result == 'success', the guard gets a legit-looking success and proceeds — the exact false negative this input is meant to catch.
Do we know what .result reports for a continue-on-error job whose step failed?
There was a problem hiding this comment.
needs.<job>.result returns success for a continue-on-error: true job whose step failed — the failure is masked and there's no job-level outcome/conclusion to read (actions/toolkit#1739). So wiring the guard to .result == 'success' would no-op it.
This isn't fixable in check.py. The guard receives a string; a masked failure and a true pass both arrive as the literal 'success'. It's well-formed, so the allow-list — which rejects failure/cancelled/typos — can't distinguish them. No logic here can.
The fix is at the source, in the consumer wiring: expose the step outcome as a job output and pass that instead of .result:
pr-maven-snapshot:
continue-on-error: true
outputs: { submitted: "${{ steps.submit.outcome }}" }
steps:
- id: submit
uses: advanced-security/maven-dependency-submission-action@...
pr-vuln-check:
with:
head-snapshot-succeeded: ${{ needs.pr-maven-snapshot.outputs.submitted || 'failure' }}
steps.submit.outcome is pre-continue-on-error, so a failed step reads failure; || 'failure' covers the job dying before the step runs. I opened a dedicated PR on the consumer side
camunda#57593) ## What Wires the dependency vulnerability gate's `head-snapshot-succeeded` guard to the **real submit-step outcome** instead of `needs.pr-maven-snapshot.result`. ## Why `pr-maven-snapshot` is `continue-on-error: true` (so a snapshot failure falls back to auto-parse without failing the run). Job-level `continue-on-error` masks a failed job to `needs.<job>.result == 'success'` — there is no job-level outcome/conclusion to read ([actions/toolkit#1739](actions/toolkit#1739)). So if the head-SBOM submission fails, `.result` still reads `success`. Feeding that to the gate's `head-snapshot-succeeded` input hands it a legit-looking success → the head side of the compare is stale/empty → a genuinely new vulnerable dependency passes silently. Harmless while the gate is observe-only; a **false green once it blocks**. The gate guard itself (camunda/infra-global-github-actions#749) is correct — it fails closed on any non-success value. It just cannot distinguish a masked-failure `'success'` from a true one; both arrive as the same string. The fix has to be at the source, in this consumer wiring. ## Change - `pr-maven-snapshot`: expose `outputs.submitted: ${{ steps.submit.outcome }}` (`id: submit` on the submit step). Step `outcome` is pre-continue-on-error, so a failed step reads `failure`. - `pr-vuln-check`: `head-snapshot-succeeded: ${{ needs.pr-maven-snapshot.outputs.submitted || 'failure' }}`. The `|| 'failure'` coalesces an empty output (job died before the submit step ran) to fail-closed. ## Ordering - Input `head-snapshot-succeeded` lands with camunda/infra-global-github-actions#749; it is inert (ignored) until that action SHA is pinned here (Renovate), so this is safe to merge before or after #749. - **Must land before** the blocking flip (camunda#57561) — otherwise blocking runs on an unguarded gate that can emit false greens with merge authority.
What
Adds a head-side verification guard to
dependency-vuln-check, mirroring the fail-closed treatment the base side already has.The gate diffs
compare(effective_base ... head_sha). The base side is resolved to a verified snapshotted ancestor and fails closed when it can't be established. The head side had no equivalent guard —check.pytrusted the compare result and failed closed only if the API call itself errored. It never verified the head SBOM was actually submitted and indexed. So:the head side read empty and a genuinely new vulnerable dependency passed silently — a false negative. Harmless while the consuming gate was observe-only; under blocking it's a false green.
Two guards
head-snapshot-succeeded. The consumer passes the result of its head-SBOM submission job (e.g.needs.pr-maven-snapshot.result == 'success'). When'false', the gate fails closed (honoring theci:vuln-gate-overridelabel, same as the base-side paths). Unset → skipped, so existing consumers are unaffected.0 added, deps removed), the compare is re-fetched a few times with backoff to let the dependency graph settle. A removal-only PR has the identical shape and simply re-confirms — the gate never blocks on this signal, it only waits out lag, so it cannot misfire into a false positive.Testing
68/68pytest pass (was 60). 8 new tests:_looks_like_missing_headhelper (3), gap-1 fail-closed + override-bypass + true-proceeds (3), gap-2 retry-then-block-on-settled-diff + persistent-empty-never-blocks (2). README + action inputs + fail-closed contract documented.Rollout
Backward compatible (new input defaults to unset/skip). After merge, the consumer (
camunda/camundaci.yml) wireshead-snapshot-succeededand Renovate bumps the pin. Companion to the vuln-gate blocking flip camunda/camunda#57561.