Skip to content

fix: stop the README capture snippet from hiding failed builds - #5

Merged
PabloCodes7 merged 1 commit into
mainfrom
fix/log-capture-footgun
Jul 14, 2026
Merged

fix: stop the README capture snippet from hiding failed builds#5
PabloCodes7 merged 1 commit into
mainfrom
fix/log-capture-footgun

Conversation

@PabloCodes7

Copy link
Copy Markdown
Contributor

The bug

The README's flagship snippet is what a maintainer copies into their workflow:

- name: Run tests
  run: pytest -q | tee test.log     # <- exits 0 even when pytest fails

GitHub's default run: shell is bash -eno pipefail — so a pipeline reports the status of its last command (tee). A failing test suite exits 0: the step goes green, if: failure() never fires, the action never runs, and the user's red build is reported as passing. Verified: bash -e -c 'false | tee x' → exit 0; bash -eo pipefail -c 'false | tee x' → exit 1.

The missing 2>&1 is the second half. Tools that report only on stderr (compilers, linters, mypy, cargo) leave the log empty, patchrail ci explain exits 2 without writing --out, and set -euo pipefail in action.yml killed the triage step with Process completed with exit code 2 — a second, meaningless red step stacked on the failure the user actually came to debug. A log-path pointing at a file that was never written was worse still: it fell through to ::error::provide either log-path or log-text, which is wrong — they did provide it.

The fix

  • README: both snippets capture with shell: bash (bash -eo pipefail) and 2>&1, plus a short section explaining why each half matters.
  • action.yml: a missing or empty log is a capture mistake, not a build failure. It now emits a warning annotation naming the cause and the correct capture recipe, sets empty outputs (guide-url = guide index), and leaves the step green. A workflow that configures no log at all still fails, as it should — the action can never work.

Guards

  • tests/test_readme_snippets.py — parses every YAML snippet in the README and fails if a capture step drops pipefail or 2>&1, or tees a file it never hands to the action. Mutation-tested: each of the three regressions turns exactly one test red.
  • tests/test_no_classification.py — the no-log path: exit 0, one-line annotation, empty outputs, summary explains the capture.
  • smoke-no-log CI job — runs the action on a real runner against an empty log and a never-written log, asserts neither fails the job. It's the shell in action.yml, not the Python, that decides whether the step survives, so this is checked end-to-end. Added to sync-v1's needs, so the v1 tag only moves when it passes.

Happy path is unchanged (verified locally: same class, confidence and guide URL).

The documented usage was `make build | tee build.log` / `pytest -q | tee
test.log`. GitHub's default `run:` shell is `bash -e` without pipefail, so a
pipeline reports `tee`'s status: a failing build exits 0, the step goes green,
`if: failure()` never fires and the action never runs. Copying the snippet made
a red build report as passing. Capture with `shell: bash` (which is
`bash -eo pipefail`) and `2>&1`, so the step stays red and stderr-only tools
land in the log.

The same missing `2>&1` also produced an empty log, where `ci explain` exits 2
and `set -euo pipefail` killed the step with a bare exit code -- a second red
step on top of the failure the user was already debugging. A missing or empty
log is now a warning annotation naming the cause and the fix, with empty outputs
and a green step; only a workflow that configures no log at all still errors.

Guards: tests/test_readme_snippets.py parses every README snippet and fails if a
capture step loses pipefail or `2>&1`, or tees a file it never triages;
tests/test_no_classification.py covers the new path; a smoke-no-log CI job runs
the action against an empty and a never-written log on a real runner, and gates
the v1 tag.
@PabloCodes7
PabloCodes7 merged commit 3441849 into main Jul 14, 2026
4 checks passed
@PabloCodes7
PabloCodes7 deleted the fix/log-capture-footgun branch July 14, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants