You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Found on PR #60, which sat MERGEABLE / CLEAN with the test matrix having never run against it once.
The problem
.github/workflows/ci.yml triggers on:
on:
pull_request:
branches: [master, main]
branches: on a pull_request trigger filters on the PR's base, not its head. So a PR opened against any branch other than master/main never runs CI at all.
That is normally fine, except for two things that combine badly:
The result is a PR that reports mergeStateStatus: CLEAN and shows a green checklist, because the checks that did run (docs.yml's build, GitGuardian) passed — while the three test jobs and tailwind-build are not pending, not failed, but absent. A green-looking PR whose tests never ran is worse than a red one.
Worked around on #60 by closing and reopening it (reopened is a default type, so that queues the full matrix). That is a manual step nobody will remember next time.
Why it matters here specifically
A docs-only diff is exactly the case where this feels safe and isn't. tests/unit/test_docs_samples.py resolves every Cf: and <c-cf.*> tag in the docs against the real catalogue and fails the build on a component that does not exist — it caught a bad tag during #54 and again during #55. That guard lives in the test matrix that a stacked docs PR skips.
Options
Drop the branches: filter from ci.yml's pull_request trigger.docs.yml already uses a bare pull_request: and is the reason docs(primitives): close Tier 3 — layout is out of scope, no grid (#55) #60 had any checks at all. Simplest fix, and it makes every PR run CI regardless of base. Cost: PRs stacked on a feature branch run the matrix twice (once on the stack, once after retarget) — cheap here, the matrix is ~2 minutes.
Add types: [opened, synchronize, reopened, edited]. Narrower, and directly targets the retarget case, but leaves the initial no-CI window open and edited also fires on title/body edits.
Both.
Option 1 alone is probably right. The branches: filter is not buying anything — there is no branch this repo wants to skip CI on.
Acceptance criteria
A PR whose base is a feature branch runs the full test matrix.
Proven non-vacuous: open a throwaway PR against a non-master base and confirm gh pr checks lists the test (3.11/3.12/3.13) jobs. The current failure mode is checks being absent, which no assertion about passing checks would catch.
If the repo ever adds required status checks in branch protection, confirm the absent-checks case blocks the merge rather than being treated as satisfied.
Found on PR #60, which sat
MERGEABLE/CLEANwith the test matrix having never run against it once.The problem
.github/workflows/ci.ymltriggers on:branches:on apull_requesttrigger filters on the PR's base, not its head. So a PR opened against any branch other thanmaster/mainnever runs CI at all.That is normally fine, except for two things that combine badly:
master— which fires apull_requestevent of typeedited. The default types forpull_requestareopened,synchronizeandreopened;editedis not among them, so no workflow run is queued.The result is a PR that reports
mergeStateStatus: CLEANand shows a green checklist, because the checks that did run (docs.yml'sbuild, GitGuardian) passed — while the threetestjobs andtailwind-buildare not pending, not failed, but absent. A green-looking PR whose tests never ran is worse than a red one.Worked around on #60 by closing and reopening it (
reopenedis a default type, so that queues the full matrix). That is a manual step nobody will remember next time.Why it matters here specifically
A docs-only diff is exactly the case where this feels safe and isn't.
tests/unit/test_docs_samples.pyresolves everyCf:and<c-cf.*>tag in the docs against the real catalogue and fails the build on a component that does not exist — it caught a bad tag during #54 and again during #55. That guard lives in the test matrix that a stacked docs PR skips.Options
branches:filter fromci.yml'spull_requesttrigger.docs.ymlalready uses a barepull_request:and is the reason docs(primitives): close Tier 3 — layout is out of scope, no grid (#55) #60 had any checks at all. Simplest fix, and it makes every PR run CI regardless of base. Cost: PRs stacked on a feature branch run the matrix twice (once on the stack, once after retarget) — cheap here, the matrix is ~2 minutes.types: [opened, synchronize, reopened, edited]. Narrower, and directly targets the retarget case, but leaves the initial no-CI window open andeditedalso fires on title/body edits.Option 1 alone is probably right. The
branches:filter is not buying anything — there is no branch this repo wants to skip CI on.Acceptance criteria
testmatrix.masterbase and confirmgh pr checkslists thetest (3.11/3.12/3.13)jobs. The current failure mode is checks being absent, which no assertion about passing checks would catch.