Skip to content

The formats shipped; nothing ran them - #107

Draft
ChelseaKR wants to merge 3 commits into
mainfrom
feat/ceqa-preflight-action
Draft

The formats shipped; nothing ran them#107
ChelseaKR wants to merge 3 commits into
mainfrom
feat/ceqa-preflight-action

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

Part of #89. The two report formats that issue asks for landed in #104; this is the packaging half — the part that makes them run without anyone remembering to. #89 stays open until you decide the two things in the last section.

What was missing

check --format sarif and --format junit exist. Nothing invokes them. The tool runs when a person runs it, which is the state the issue describes: "Today the tool runs when someone remembers."

What changed

action.yml — a composite action. It installs a pinned release wheel, checks the packages in JSON, SARIF and JUnit, uploads the SARIF to code scanning, writes the counts to the job summary, and exits on the tool's own code (0 clean, 1 findings, 2 a package it could not read).

version is required and has no default. A floating tag would make a report's tool_version unpredictable, and that field is read as evidence about which catalog produced the findings. The install step also compares the tag against the version it actually installed, and refuses a release carrying more than one wheel rather than picking by sort order.

ceqa_preflight.ci_summary — the module behind the failures, warnings and not-run outputs. It calls the same summarize_counts the console summary line calls. Counting in shell would have been four lines shorter and would have been a second implementation of a published number, drifting from the first one with nothing in the repository to notice. Its other job is the failure mode: a report directory holding nothing exits 2 and says "no report produced", rather than publishing failures=0 for a job that checked nothing.

.pre-commit-hooks.yaml and a pre-commit command. pre-commit hands a hook the staged paths, not the package they belong to. The tempting fallback — the working directory, or the files' bare common parent — is how a hook becomes a check that cannot fail: it reports on a package nobody is filing and passes while doing it. So resolution walks up from the staged files' common directory, no higher than where the hook was invoked, to the first directory holding a package.yaml / package.yml / package.json, and takes the filing type from that manifest rather than defaulting one — a default would check an NOD package against the NOE rules and pass. When no package can be identified it exits 2 and asks for args: [--package, ...].

check's body moves into _check_sources so both commands run the same loop. Nothing on the check path behaves differently.

The one thing worth knowing before you read the diff

The action cannot run today. It installs the wheel from a GitHub Release, and this repository has cut no tag — git ls-remote --tags is empty, which is #85. Both the README and the example workflow now say so in terms and point at the release-status section, so the uses: ChelseaKR/ceqa-preflight@v0.1.0 line reads as the shape of the reference rather than a version anyone can pin. A second commit on this branch does only that; without it the README would document an install that fails, which is the drift class this repository keeps finding in its own prose.

How it was verified

make verify clean — lock-check lint typecheck test security audit i18n, the audit reaching the network. 564 tests, 0 failures. README and docs/I18N.md gate figures updated to 564 / 40 / 211, and the one new console string is translated, so make i18n reports 211 messages at parity across en and es.

action.yml, .pre-commit-hooks.yaml and the example workflow are YAML that CI here cannot execute, so the tests assert the parts that rot silently: that every flag the action passes still exists on check, that the outputs it declares are exactly the keys ci_summary writes, that version still has no default and the install step still compares versions, and that the example grants security-events: write — which the caller must, because a composite action cannot widen the token it is handed.

Four negative controls, each replacing a literal, each read back from disk and SHA-256-compared before running and restored from a cp -p copy compared again after:

sabotage landed result
the empty-directory refusal returns [] instead of raising 525bfa6e…00639597… RED — 1 test; the sabotaged run prints not-run=0 for a directory with no report, which is the whole point
the hook falls back to the working directory instead of refusing 086a2a1e…1216cad0… RED — 1 test
the action's last step becomes exit 0 a48ed3e5…be58d79c… RED — 1 test
warnings reads the failure count 525bfa6e…c66c99ae… RED — 2 tests

The fourth is the one that says whether ci_summary is worth having. It is a one-word change with no syntax error and no type error, exactly the shape a shell reimplementation would have drifted into, and it reddens the test that compares the published counts against summarize_counts on a real report.

The first control is worth reading for what it printed rather than for the failure: with the refusal removed the command emitted failures=0 warnings=0 not-run=0 and exited 0, so the action would have uploaded an empty SARIF, written a table of zeroes to the job summary, and gone green on a run that produced nothing.

What needs your call

  1. Publishing an Action is a distribution claim. action.yml at the repository root makes uses: ChelseaKR/ceqa-preflight@<tag> a public interface, and interfaces are harder to withdraw than to add. The issue's scope names the action explicitly and puts Marketplace publication out of scope, which is what I built, but the file at the root is the part you may want to sit with. It is inert until Cut the first tagged release #85 in any case.

  2. The pinned codeql-action/upload-sarif SHA is cdf488f595d80d6e07e03d4674febd5ab45fa938 (v4.37.9) — the same one .github/workflows/security.yml already pins. Dependabot updates this repository's own workflows; whether it also updates a consumer-facing action.yml, and whether you want it to, is a policy question I did not settle.

Not built, so #89 keeps them: nothing here publishes to the Marketplace, and nothing runs the ai commands in CI — they need credentials and send text off-machine, which the issue puts out of scope and which I agree with.

Prepared with AI assistance; reviewed before submission.

`check` gained SARIF and JUnit in #104, and the tool still runs when someone
remembers to run it. This is the packaging half of #89: a composite action, a
pre-commit hook, and an example workflow.

`action.yml` installs a pinned release wheel, checks the packages, uploads the
SARIF to code scanning, writes the counts to the job summary, and exits on the
tool's own code. `version` is required and has no default: a floating tag would
make a report's `tool_version` unpredictable, and that field is read as
evidence about which catalog ran, so the install step also compares the tag
against the version it actually installed.

The action's `failures`, `warnings` and `not-run` outputs come from
`ceqa_preflight.ci_summary`, which calls the same `summarize_counts` the
console summary line calls. Counting in shell would have been shorter and would
have been a second implementation of a published number, drifting from the
first one with nothing to notice. It exits 2 on a directory holding no report,
rather than publishing `failures=0` for a job that checked nothing.

`.pre-commit-hooks.yaml` adds a `ceqa-preflight` hook, and a `pre-commit`
command behind it. `pre-commit` hands a hook the staged paths, not the package
they belong to, and the tempting fallback -- the working directory, or the
files' common parent -- is how a hook becomes a check that cannot fail:
checking the wrong directory reports on a package nobody is filing and passes
while doing it. So resolution walks up from the staged files' common directory,
no higher than where the hook was invoked, to the first directory holding a
manifest, and takes the filing type from that manifest rather than defaulting
one. When no package can be identified it exits 2 and asks for `--package`.

`check`'s body moves into `_check_sources` so both commands run the same loop.
No behaviour changes on the `check` path.
…agged

The README and the example workflow both wrote `uses: ChelseaKR/ceqa-preflight@v0.1.0`.
No tag exists on this repository, so that line documents an install that fails. Both now
say so and point at the release-status section.
@ChelseaKR
ChelseaKR force-pushed the feat/ceqa-preflight-action branch from 040963a to 5fd8fde Compare September 7, 2026 06:48
`test_every_flag_the_action_passes_still_exists_on_the_cli` scraped `--help`
output. Rich wraps and colours that at the terminal's width, so on an
80-column runner `--filing-type` arrives split across two lines and the
substring is absent. Green on a wide developer terminal, red on all three CI
platforms -- a test measuring the runner rather than the code, which is the one
failure mode a gate must not have.

It now reads the option strings off the command objects `typer.main.get_command`
returns, which is both width-independent and a stronger assertion: it sees the
real parameter names rather than their rendering.

Verified the replacement still fails for the reason it exists: renaming
`--filing-type` to `--filing-kind` in all three of its declarations reddens it
(and the SARIF/JSON agreement test), and restoring the file byte-for-byte turns
both green.
@ChelseaKR

Copy link
Copy Markdown
Owner Author

Rebased onto main (e8d390d) after #106 merged, and one real defect of my own is fixed on top.

The defect. test_every_flag_the_action_passes_still_exists_on_the_cli scraped --help output to check that the flags action.yml passes still exist. Rich wraps and colours that at the terminal's width, so on an 80-column runner --filing-type arrives split across two lines and the substring is simply absent. It was green on my terminal and red on all three CI platforms — a test measuring the runner rather than the code, which is the one failure mode a gate must not have. It now reads the option strings off the command objects typer.main.get_command returns: width-independent, and a stronger assertion, since it sees the real parameter names rather than their rendering.

Verified the replacement still fails for the reason it exists rather than merely passing: renaming --filing-type to --filing-kind in all three declarations reddens it, and restoring cli.py byte-for-byte (SHA-256 3c18fae9… before and after) turns it green. Also confirmed green at COLUMNS=40.

The rebase. Four conflicts, all mechanical: cli.py imports (both blocks belong), the CHANGELOG.md Unreleased position (both entries kept), and the README / docs/I18N.md gate figures. The figures were recomputed, not merged — 590 tests, 41 source files, 222 catalogue messages — because git will happily auto-merge two contradictory numbers into a third that is neither. The i18n catalogs were reset to main's and regenerated rather than resolved by hand, so the compiled .mo files match their .po sources.

make verify clean on the rebased head: lock-check lint typecheck test security audit i18n, the audit reaching the network. 590 passed.

Still a draft, for the two reasons in the body: publishing action.yml at the repository root makes uses: ChelseaKR/ceqa-preflight@<tag> a public interface, and the action is inert until #85 cuts the first tag.

One sequencing note, since both open pull requests here are drafts waiting on you: #105 and this branch overlap only in CHANGELOG.md and README.md. Whichever lands second will need its README gate figures recomputed against the merged tree — the numbers, not the prose, are the whole conflict.

Prepared with AI assistance; reviewed before submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant