Skip to content

fix: refuse a ci-result schema the action cannot read, and bound the patchrail install - #6

Merged
PabloCodes7 merged 1 commit into
mainfrom
fix/ci-result-schema-guard
Jul 14, 2026
Merged

fix: refuse a ci-result schema the action cannot read, and bound the patchrail install#6
PabloCodes7 merged 1 commit into
mainfrom
fix/ci-result-schema-guard

Conversation

@PabloCodes7

Copy link
Copy Markdown
Contributor

The bug

The action installs patchrail unpinned, and scripts/annotate.py reads every ci-result field with .get(). So a patchrail release that changes the ci-result contract lands in every consumer's CI with no commit here to review — and it does not raise. It degrades:

::warning title=PatchRail CI Triage::unknown (confidence None) — guide: https://getpatchrail.com/fix
failure-class=unknown
confidence=None
guide-url=https://getpatchrail.com/fix

That is a confident-looking PatchRail annotation, on a run that is already red, that says nothing — under a @v1 tag users pinned precisely so nothing would move under them.

And it was silent: the smoke assertions (test -n on the class, a fix* glob on the URL) both pass on exactly that output, so CI would have stayed green while every user saw garbage. Not hypothetical — patchrail 0.4.0 moved ci classes to schema v2 in a minor bump.

The fix

  • annotate.py checks schema_version and, when it is not the contract it reads, names both versions and the way out (patchrail-version, or upgrade the action) instead of inventing a classification. Empty outputs, exit 0 — it runs under if: failure(), where a second red step buries the failure the user actually came to debug.
  • action.yml installs patchrail>=0.3.1,<0.5.0 by default (verified: 0.3.1 and 0.4.0 both emit patchrail.ci_result.v1), so a breaking release cannot reach users on its own. Pinning an exact version via patchrail-version is unchanged.
  • latest-patchrail job runs the real explain -> annotate path against the newest patchrail on PyPI, every push and weekly. A breaking release now turns this repo red before it turns users red. It is deliberately not a gate on sync-v1: the guide-slug job installs the shipped range instead (grepped out of action.yml so the two cannot drift), so an upstream break can never stop merged fixes from reaching @v1.
  • Smoke assertions now check the class the sample log actually is, so a degraded result cannot pass them.

Verification

  • Suite: 23 passed on Python 3.12 (the CI version), up from 16.
  • Mutation-tested both ways: deleting the guard → 4 red; loosening it to accept any schema → 4 red; restored → 23 green.
  • End-to-end against patchrail 0.4.0: still classifies python_test_failure (0.89) → /fix/python-test-failure.
  • End-to-end against a simulated future ci_result.v2: no invented class, empty outputs, message naming both schemas.

…patchrail install

The action installed patchrail unpinned, and `annotate.py` read every ci-result
field with `.get()`. A patchrail release that changes the ci-result contract
therefore lands in every consumer's CI with no commit here, and does not raise:
it degrades to `unknown (confidence None)` pointing at the bare guide index, on a
run that is already red, under a `@v1` tag pinned precisely so nothing would move.

It was also silent. The smoke assertions (`test -n` on the class, a `fix*` glob on
the URL) both pass on that output, so CI would have stayed green while users saw
meaningless annotations. This is not hypothetical: patchrail 0.4.0 moved
`ci classes` to schema v2 in a minor bump.

- annotate.py checks `schema_version` and, when it is not the one it reads, names
  both versions and the way out instead of inventing a classification. Empty
  outputs, exit 0: it runs under `if: failure()`, so a second red step would bury
  the failure the user came to see.
- action.yml installs `patchrail>=0.3.1,<0.5.0` by default (verified: both emit
  `patchrail.ci_result.v1`), so a breaking release cannot reach users on its own.
- The guide-slug job installs that same range, grepped out of action.yml so the
  two cannot drift, and the new `latest-patchrail` job runs the real
  explain -> annotate path against the newest release on PyPI. A breaking release
  now turns this repo red before it turns users red, without blocking `@v1`.
- The smoke assertions check the class the sample log actually is, so a degraded
  result can no longer pass them.
@PabloCodes7
PabloCodes7 merged commit 7833871 into main Jul 14, 2026
5 checks passed
@PabloCodes7
PabloCodes7 deleted the fix/ci-result-schema-guard branch July 14, 2026 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant