Skip to content

ci(release): check the release people can actually download - #146

Merged
donislawdev merged 2 commits into
masterfrom
ci/verify-published-release
Aug 21, 2026
Merged

ci(release): check the release people can actually download#146
donislawdev merged 2 commits into
masterfrom
ci/verify-published-release

Conversation

@donislawdev

Copy link
Copy Markdown
Owner

The verify command in both READMEs did not work, and 0.5.0 shipped it.

$ gh attestation verify BeanNetworkTester-v0.5.0-windows-x64.zip -R donislawdev/BeanNetworkTester
Error: HTTP 404: Not Found (.../attestations/sha256:f6b8bbfe...?predicate_type=https%3A%2F%2Fslsa.dev%2Fprovenance%2Fv1)

gh defaults to asking for build provenance. The archive a user downloads deliberately has
none: a person signs it on their own machine, and attest-release.yml refuses to claim a
runner built it. What ships beside the release is the SBOM attestation over the signed bytes,
so the command has to say so. Measured, not assumed - adding
--predicate-type https://spdx.dev/Document/v2.3 makes both the online and the offline form
exit 0 against the real published release.

This is the second time that command has been wrong, and both times a person found it after
the fact, because nothing in the suite runs gh. So the text is only half the fix.

What changed

The documentation

  • All three occurrences across README.md and README.pl.md now name the predicate type, and
    the prose no longer promises a proof the file does not carry.

A workflow that checks what people download (verify-release.yml, on release: published)

  • Runs the README's own commands, verbatim, against the published assets - not equivalents, so
    the documentation cannot drift away from something that works.
  • Then: all four assets present, sha256sum -c, Get-AuthenticodeSignature with status Valid
    and a timestamp, the signing certificate's SHA-256 against the pin read out of
    beantester/legal.py, and, for a full release, that /releases/latest is this tag - the
    site's download button points there.
  • contents: read and nothing else. Re-runnable by workflow_dispatch for any published tag.

Two manual rules become gates

  • release.yml refuses to build unless CI succeeded on exactly the tagged commit. It builds and
    publishes but does not test, so this was the last large step of the recipe kept by memory. It
    asks for the CI workflow by name rather than for a clean sweep, because one check in this
    repository fails for a licensing reason of its own and would otherwise block every release.
  • sign_release.py acts on the certificate's expiry instead of printing it: a warning inside 90
    days, a refusal after it, both naming CODESIGN_SHA256 because a renewal is a different
    certificate and the pin moves with it.

And a seventh step for phase B

  • sign_release.py now waits for the attestation workflow and confirms the draft is complete:
    four assets, the checksums downloaded back from the release naming the digest that was
    actually signed, still a draft. It used to end at "dispatched, go look" - and a draft missing
    one file looks almost exactly like a finished one.

Verification

  • tests/test_version_and_release.py - 28 passed (25 before; three new guards).
  • The widened guard on the documented commands found the third occurrence in each README within
    a minute of being written, which is the shape of guard worth having.
  • Both documented commands run against the published v0.5.0: exit 0 with the predicate type,
    HTTP 404 without it.
  • The PowerShell step's logic run over the real published archive: Valid, timestamped by
    Certum, certificate SHA-256 equal to the pin.
  • All three release workflows parse as YAML; the new one triggers on release: published,
    holds contents: read, and runs on Windows because Authenticode exists nowhere else.
  • ruff and mypy clean on the changed Python.

The full suite runs on CI rather than locally.

donislawdev and others added 2 commits August 21, 2026 13:15
The verify command in both READMEs did not work, and 0.5.0 shipped it. `gh
attestation verify <zip> -R donislawdev/BeanNetworkTester` answers HTTP 404: it
defaults to asking for build provenance, and the archive a user downloads
deliberately has none, because a person signs it on their own machine and the
attestation workflow refuses to claim a runner built it. What travels with the
release is the SBOM attestation over the signed bytes, so the command has to say
so. All three occurrences across the two READMEs now name the predicate type, and
the prose no longer promises a proof the file does not carry.

That is the second time this command has been wrong, and both times a human found
it afterwards, because nothing in the suite runs `gh`. So the fix is not only the
text:

- A new workflow runs on `release: published` and executes the README's own
  commands, verbatim, against the published assets - then the checksums, the
  Authenticode signature, the presence of a timestamp, the signing certificate
  against the pin in beantester/legal.py, and, for a full release, that
  /releases/latest is this tag, because the site's download button points there.
  It holds `contents: read` and nothing else.
- The existing guard now reads EVERY documented verify command instead of only
  the one containing `--bundle`. It found the third occurrence in each README a
  minute after being widened.

Two more manual rules become gates while the subject is open:

- release.yml refuses to build unless CI succeeded on exactly the tagged commit.
  It builds and publishes but does not test, so this was the last large step of
  the recipe kept by memory alone. It asks for the CI workflow by name rather
  than for a clean sweep of every check, because one check here fails for a
  licensing reason of its own and would otherwise block every release.
- sign_release.py acts on the certificate's expiry date instead of printing it: a
  warning inside 90 days, a refusal after it, both naming CODESIGN_SHA256,
  because a renewal is a different certificate and the pin moves with it.
- sign_release.py also gained a seventh step that waits for the attestation
  workflow and confirms the draft is complete - four assets, the published
  checksums naming the digest that was signed, still a draft. It used to end at
  "dispatched, go look".

Verified against the published v0.5.0 rather than in the abstract: both README
commands exit 0 with the predicate type and 404 without it, and the PowerShell
step's logic run over the real archive reports Valid, timestamped, and a
certificate matching the pin.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…mmand

Two guards caught the previous commit, and both were right.

The mutation registry pins the documented verify command by removing its
predicate type and expecting a test to redden. Its pattern was the flag on its
own, which matched exactly once while only one command carried it. Adding the
flag to the other two occurrences made that pattern match three places, and a
pattern that matches three places proves nothing about any of them. There are two
entries now, anchored on `.sigstore.json` and on `-R` - each unique, each
version-independent, one for the offline command and one for the online one that
actually broke. Both were run for real and both are caught.

The other is the no-semicolons rule for README prose: the sentence explaining why
`--predicate-type` is required used one. It is two sentences now, in both
languages.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@donislawdev
donislawdev merged commit 1d759d1 into master Aug 21, 2026
13 checks passed
@donislawdev
donislawdev deleted the ci/verify-published-release branch August 21, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant