test: fail CI when the fix-guide slug list drifts - #3
Merged
Conversation
FIX_GUIDE_SLUGS is hand-maintained, and its comment still claimed to mirror patchrail.cli._FIX_GUIDE_SLUGS -- a symbol the CLI removed, so nothing had tied the list to reality for a while. It can rot two ways: a slug that is not a real failure class (dead entry, or a rename that silently drops the guide link), or a slug with no published guide (the action hands users a 404). Derive both sources of truth -- 'patchrail ci classes' and the guide index on getpatchrail.com -- and fail on either divergence. The network check skips cleanly if the site is unreachable, so an outage cannot red the repo. Also run weekly: publishing or removing a guide drifts the list with no commit here. Today the 31 slugs match the 31 published guides exactly; the 9 newer classes correctly fall back to the index.
PabloCodes7
added a commit
that referenced
this pull request
Jul 14, 2026
Everyone pins patchrail/ci-triage-action@v1, but CI only ever tested the commit. The tag had fallen two commits behind main, so the guide-slug guard added in #3 was validating code that no user actually runs. Move v1 to each main commit that passes both jobs, and document in the README that v1 floats within the v1 line (with SHA pinning as the alternative). Co-authored-by: PabloCodes7 <pablocodes7@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FIX_GUIDE_SLUGSinscripts/annotate.pydecides whether a failure class links to its/fix/<slug>guide or to the generic index. It is hand-maintained, and its comment claimed to mirrorpatchrail.cli._FIX_GUIDE_SLUGS— a symbol the CLI has since removed, so nothing actually tied the list to reality.It can rot in two directions:
This derives both sources of truth (
patchrail ci classesand the guide index on getpatchrail.com) and fails on either divergence. Mutation-checked: adding a bogus slug, or dropping one that is published, both turn the suite red.Current state is clean — the 31 slugs match the 31 published guides exactly, and the 9 newer classes (xcode, docs-build, shell-lint, elixir, db-migration, k8s, helm, node-script-missing, pre-commit) correctly fall back to the index rather than 404.
Notes: the network check skips rather than fails if the site is unreachable, so an outage can't red the repo. It also runs weekly, since publishing or removing a guide drifts the list with no commit here to trigger CI.