From 42d53a21050d672f00bbc6dbebd194628126b8e2 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Fri, 21 Aug 2026 13:15:42 +0200 Subject: [PATCH 1/2] ci(release): check the release people can actually download The verify command in both READMEs did not work, and 0.5.0 shipped it. `gh attestation verify -R donislawdev/BeanNetworkTester` answers HTTP 404: it defaults to asking for build provenance, and the archive a user downloads deliberately has none, because a person signs it on their own machine and the attestation workflow refuses to claim a runner built it. What travels with the release is the SBOM attestation over the signed bytes, so the command has to say so. All three occurrences across the two READMEs now name the predicate type, and the prose no longer promises a proof the file does not carry. That is the second time this command has been wrong, and both times a human found it afterwards, because nothing in the suite runs `gh`. So the fix is not only the text: - A new workflow runs on `release: published` and executes the README's own commands, verbatim, against the published assets - then the checksums, the Authenticode signature, the presence of a timestamp, the signing certificate against the pin in beantester/legal.py, and, for a full release, that /releases/latest is this tag, because the site's download button points there. It holds `contents: read` and nothing else. - The existing guard now reads EVERY documented verify command instead of only the one containing `--bundle`. It found the third occurrence in each README a minute after being widened. Two more manual rules become gates while the subject is open: - release.yml refuses to build unless CI succeeded on exactly the tagged commit. It builds and publishes but does not test, so this was the last large step of the recipe kept by memory alone. It asks for the CI workflow by name rather than for a clean sweep of every check, because one check here fails for a licensing reason of its own and would otherwise block every release. - sign_release.py acts on the certificate's expiry date instead of printing it: a warning inside 90 days, a refusal after it, both naming CODESIGN_SHA256, because a renewal is a different certificate and the pin moves with it. - sign_release.py also gained a seventh step that waits for the attestation workflow and confirms the draft is complete - four assets, the published checksums naming the digest that was signed, still a draft. It used to end at "dispatched, go look". Verified against the published v0.5.0 rather than in the abstract: both README commands exit 0 with the predicate type and 404 without it, and the PowerShell step's logic run over the real archive reports Valid, timestamped, and a certificate matching the pin. Co-authored-by: Claude Opus 5 --- .github/workflows/release.yml | 35 ++++++ .github/workflows/verify-release.yml | 164 ++++++++++++++++++++++++++ CHANGELOG.md | 9 ++ README.md | 28 +++-- README.pl.md | 28 +++-- tests/test_version_and_release.py | 127 ++++++++++++++++++++ tools/sign_release.py | 167 +++++++++++++++++++++++++-- 7 files changed, 528 insertions(+), 30 deletions(-) create mode 100644 .github/workflows/verify-release.yml diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 77fbe62..7087a5e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -40,9 +40,44 @@ jobs: # above it. id-token: write attestations: write + # Reading the CI runs for the tagged commit, for the gate that is the first + # step below. Read-only, and it is the whole reason this scope is here. + actions: read steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + # 馃敶 FIRST, before anything is installed or built: was CI green on EXACTLY + # this commit? This workflow deliberately does not re-run the tests - it + # builds and publishes - so until 2026-08-21 the only thing standing between a + # red commit and a published release was somebody remembering. That was step 1 + # of the recipe, it was written down as manual, and a rule kept by memory is a + # rule that eventually is not. + # + # Keyed on the CI workflow BY NAME rather than "are all checks green": this + # repository has a check that fails for a reason of its own (a code-scanning + # feature the account is not licensed for), and a gate that demands a clean + # sweep of everything would block every release over something unrelated. + - name: Refuse to build unless CI was green on this commit + shell: bash + env: + GH_TOKEN: ${{ github.token }} + run: | + runs="$(gh api \ + "repos/$GITHUB_REPOSITORY/actions/runs?head_sha=$GITHUB_SHA&status=completed" \ + --jq '[.workflow_runs[] | select(.name == "CI")] | map(.conclusion)')" + echo "completed CI runs on $GITHUB_SHA: $runs" + if [ "$runs" = "[]" ]; then + echo "::error::No completed CI run for $GITHUB_SHA. Tag a commit that CI has" + echo "::error::finished on - this workflow does not run the tests itself." + exit 1 + fi + if ! echo "$runs" | grep -q '"success"'; then + echo "::error::CI did not succeed on $GITHUB_SHA ($runs). Releasing this tag" + echo "::error::would publish code that was never proven to build or pass." + exit 1 + fi + echo "CI is green on the commit being released." + # The interpreter PyInstaller freezes into the shipped bundle. Keep it in # step with the build job in ci.yml, or CI smoke-tests one artefact and # users download another. diff --git a/.github/workflows/verify-release.yml b/.github/workflows/verify-release.yml new file mode 100644 index 0000000..848cdf8 --- /dev/null +++ b/.github/workflows/verify-release.yml @@ -0,0 +1,164 @@ +# The last step of a release: check the thing people can actually download. +# +# Everything before this verifies an artefact, a draft, or a digest passed between +# workflows. None of it touches the published release page, and the published page is +# the only thing a user ever sees. Until 2026-08-21 the commands in the README were +# run by a human, occasionally, after the fact - and that is exactly how the first +# documented verify command shipped broken for every user of 0.5.0-rc.2 and again +# for 0.5.0, where `gh attestation verify -R ` answers HTTP 404 because +# it looks for build provenance that a hand-signed archive deliberately does not have. +# +# 馃敶 So this runs THE COMMANDS FROM THE README, verbatim, against the published +# assets. Not equivalents, not a re-implementation: if the README's command stops +# working, this has to be what goes red. `tests/test_version_and_release.py` pins the +# two together so neither can drift alone. +# +# Runs on Windows because one of the checks is Authenticode, which does not exist +# anywhere else - and the signature is the half of the release a checksum cannot +# speak for. +name: Verify the published release + +on: + release: + types: [published] + # Re-runnable by hand against any published tag, because "did that old release + # still verify" is a question worth being able to ask without cutting a new one. + workflow_dispatch: + inputs: + tag: + description: "Published tag to check, e.g. v0.5.0" + required: true + +permissions: + contents: read + +jobs: + verify: + name: Download what users download, then check it + runs-on: windows-latest + timeout-minutes: 15 + permissions: + # Reading the release assets. Nothing here writes anything, anywhere. + contents: read + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - name: Work out which release we are checking + id: which + shell: bash + env: + EVENT_TAG: ${{ github.event.release.tag_name }} + INPUT_TAG: ${{ inputs.tag }} + PRERELEASE: ${{ github.event.release.prerelease }} + run: | + tag="${EVENT_TAG:-$INPUT_TAG}" + test -n "$tag" || { echo "no tag to check"; exit 1; } + echo "tag=$tag" >> "$GITHUB_OUTPUT" + # A dispatch has no event payload, so assume "not a pre-release" only when + # the payload said so; otherwise ask the API below rather than guessing. + echo "prerelease=${PRERELEASE:-unknown}" >> "$GITHUB_OUTPUT" + echo "checking $tag (prerelease=${PRERELEASE:-unknown})" + + - name: Download every published asset + shell: bash + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ steps.which.outputs.tag }} + run: | + mkdir -p published && cd published + gh release download "$TAG" --repo "$GITHUB_REPOSITORY" + ls -l + # Four assets ship: the archive, the checksums, the SBOM and the bundle. + # A missing one means a phase did not finish, which is precisely the + # half-failure nobody would notice by looking at the page. + for pattern in '*.zip' 'SHA256SUMS.txt' '*.spdx.json' '*.sigstore.json'; do + # shellcheck disable=SC2086 + ls $pattern >/dev/null 2>&1 || { echo "MISSING asset: $pattern"; exit 1; } + done + + - name: The checksum users are told to compare + shell: bash + working-directory: published + run: | + # The file is written in `sha256sum -c` format; the `*` before the name is + # the binary-mode marker, not a typo, and `sha256sum` expects it. + sha256sum -c SHA256SUMS.txt + + - name: The offline command from the README, verbatim + shell: bash + working-directory: published + env: + GH_TOKEN: ${{ github.token }} + run: | + zip="$(ls ./*.zip)" + bundle="$(ls ./*.sigstore.json)" + gh attestation verify "$zip" \ + --bundle "$bundle" \ + --repo "$GITHUB_REPOSITORY" \ + --predicate-type https://spdx.dev/Document/v2.3 + + - name: The online command from the README, verbatim + shell: bash + working-directory: published + env: + GH_TOKEN: ${{ github.token }} + run: | + zip="$(ls ./*.zip)" + gh attestation verify "$zip" \ + -R "$GITHUB_REPOSITORY" \ + --predicate-type https://spdx.dev/Document/v2.3 + + - name: The signature, and that it is OUR certificate + shell: pwsh + working-directory: published + run: | + # Convention 46: pwsh first. This step is the reason the job runs on + # Windows - a checksum says the bytes are unchanged, and says nothing at + # all about who signed them. + $zip = (Get-ChildItem -Filter *.zip)[0].FullName + Expand-Archive -LiteralPath $zip -DestinationPath unpacked -Force + $exe = Get-ChildItem -Path unpacked -Filter BeanNetworkTester.exe -Recurse | + Select-Object -First 1 + if (-not $exe) { throw "no BeanNetworkTester.exe inside the archive" } + + $sig = Get-AuthenticodeSignature -LiteralPath $exe.FullName + "status : $($sig.Status)" + "subject : $($sig.SignerCertificate.Subject)" + if ($sig.Status -ne 'Valid') { throw "signature status is $($sig.Status)" } + + # Without a timestamp the signature dies when the certificate expires, so + # its absence is a real defect and not a detail. + if (-not $sig.TimeStamperCertificate) { throw "the signature carries no timestamp" } + "stamped : $($sig.TimeStamperCertificate.Subject)" + + $bytes = [System.Security.Cryptography.SHA256]::Create().ComputeHash( + $sig.SignerCertificate.RawData) + $actual = ($bytes | ForEach-Object { $_.ToString('x2') }) -join '' + # Read the pin out of the source rather than repeating it here: two copies + # of a constant are two things that can disagree, and this one moves when + # the certificate is renewed. + $line = Select-String -Path ../beantester/legal.py -Pattern '^CODESIGN_SHA256 = "([0-9a-f]{64})"' + if (-not $line) { throw "could not read CODESIGN_SHA256 out of beantester/legal.py" } + $pinned = $line.Matches[0].Groups[1].Value + "pinned : $pinned" + "actual : $actual" + if ($actual -ne $pinned) { throw "the shipped file was signed by a DIFFERENT certificate" } + + - name: A full release must be the one the website offers + shell: bash + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ steps.which.outputs.tag }} + run: | + # The download button on the project site points at /releases/latest, so + # "published" and "what people get" are only the same thing if this holds. + # Asked of the API rather than the event payload, because a dispatch has no + # payload and a pre-release is SUPPOSED to fail this. + prerelease="$(gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json isPrerelease --jq .isPrerelease)" + if [ "$prerelease" = "true" ]; then + echo "$TAG is a pre-release: it must NOT be latest, and is not checked here" + exit 0 + fi + latest="$(gh api "repos/$GITHUB_REPOSITORY/releases/latest" --jq .tag_name)" + echo "latest is $latest, this release is $TAG" + test "$latest" = "$TAG" diff --git a/CHANGELOG.md b/CHANGELOG.md index 44d0480..723616d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,15 @@ The format follows [Keep a Changelog](https://keepachangelog.com/); versions fol ## [Unreleased] +### Fixed + +- **The command for checking your download did not work.** Both READMEs told you to run + `gh attestation verify -R donislawdev/BeanNetworkTester`, and it answers `HTTP 404`. + It looks for proof the file was built on a runner, and the file you download is signed on + the maintainer's own machine, so what travels with it is proof of the bill of materials + instead. The command now says which proof to ask for, and every published release is + checked with the exact commands the README gives you, so a broken one cannot ship again. + ## [0.5.0] - 2026-08-20 **The short version.** Three things, and the first is about not losing your files when the diff --git a/README.md b/README.md index 33308f1..d157b04 100644 --- a/README.md +++ b/README.md @@ -1495,17 +1495,23 @@ changes is that the warning now says who signed it. The **WinDivert driver itsel signed by its author**. You can still compare the release's SHA-256 checksum (`SHA256SUMS.txt`) to confirm the file arrived unchanged. -**You can also check where the download came from, not just that it is unchanged.** Every release -archive carries a signed build attestation, so one command answers "was this really built from that -source by that workflow": +**You can also check what the download contains, not just that it is unchanged.** Every release +archive carries a signed statement listing everything inside it, made by this repository's own +workflow over the exact bytes you downloaded: ```bash -gh attestation verify BeanNetworkTester-v0.5.0-windows-x64.zip -R donislawdev/BeanNetworkTester +gh attestation verify BeanNetworkTester-v0.5.0-windows-x64.zip -R donislawdev/BeanNetworkTester --predicate-type https://spdx.dev/Document/v2.3 ``` -A checksum proves the file matches what the release page says. This proves the release page itself -was produced by this repository's own workflow, from a specific commit, on a GitHub-hosted runner. -The same command also verifies the SBOM that ships beside the archive. +A checksum proves the file matches what the release page says. This proves the bill of materials +beside it describes those same bytes, and that the statement was made by a workflow in this +repository rather than by whoever handed you the file. + +`--predicate-type` is not optional. Without it `gh` looks for a build-provenance statement and +answers `HTTP 404`, because the archive you download was **signed on the maintainer's machine**, +not built on a runner - the signing key is on a card that no runner can reach. Provenance is +attested for the unsigned build inside the workflow; what travels with the release is the bill of +materials over the signed file. That command asks GitHub which attestations exist. The proof also ships **as a file**, `BeanNetworkTester-vX.Y.Z.sigstore.json`, so you can check the archive against evidence that @@ -1537,8 +1543,10 @@ component in the build with its version, licence and where its source lives. It The SBOM is **signed against the archive it describes**, so the two cannot be separated: ```bash -gh attestation verify BeanNetworkTester-vX.Y.Z-windows-x64.zip --repo donislawdev/BeanNetworkTester +gh attestation verify BeanNetworkTester-vX.Y.Z-windows-x64.zip --repo donislawdev/BeanNetworkTester --predicate-type https://spdx.dev/Document/v2.3 ``` -A checksum tells you the file arrived unchanged. The attestation tells you that *this* build, -with *this* bill of materials, came out of this repository's release workflow. +A checksum tells you the file arrived unchanged. The attestation tells you that *this* bill of +materials belongs to *this* archive, and that the statement was made by this repository's release +workflow. Leave `--predicate-type` out and `gh` looks for a build-provenance statement instead and +answers `HTTP 404`. diff --git a/README.pl.md b/README.pl.md index 0a9da8b..1b11dd0 100644 --- a/README.pl.md +++ b/README.pl.md @@ -1350,17 +1350,23 @@ Sam sterownik **WinDivert jest podpisany cyfrowo przez jego autora**. Sum臋 kontroln膮 SHA-256 wydania (`SHA256SUMS.txt`) nadal mo偶esz por贸wna膰, 偶eby potwierdzi膰, 偶e plik dotar艂 bez zmian. -**Mo偶esz te偶 sprawdzi膰, sk膮d ten plik pochodzi, a nie tylko czy si臋 nie zmieni艂.** Ka偶de archiwum -wydania niesie podpisan膮 atestacj臋 builda, wi臋c jedno polecenie odpowiada na pytanie 鈥瀋zy to -naprawd臋 zbudowano z tego kodu, tym workflow": +**Mo偶esz te偶 sprawdzi膰, co ten plik zawiera, a nie tylko czy si臋 nie zmieni艂.** Ka偶de archiwum +wydania niesie podpisany spis wszystkiego, co jest w 艣rodku, zrobiony przez workflow tego +repozytorium nad dok艂adnie tymi bajtami, kt贸re pobra艂e艣: ```bash -gh attestation verify BeanNetworkTester-v0.5.0-windows-x64.zip -R donislawdev/BeanNetworkTester +gh attestation verify BeanNetworkTester-v0.5.0-windows-x64.zip -R donislawdev/BeanNetworkTester --predicate-type https://spdx.dev/Document/v2.3 ``` -Suma kontrolna dowodzi, 偶e plik zgadza si臋 z tym, co m贸wi strona wydania. To dowodzi, 偶e sama -strona wydania powsta艂a z workflow tego repozytorium, z konkretnego commita, na maszynie GitHuba. -Tym samym poleceniem sprawdzisz te偶 SBOM, kt贸ry jedzie obok archiwum. +Suma kontrolna dowodzi, 偶e plik zgadza si臋 z tym, co m贸wi strona wydania. To dowodzi, 偶e spis +sk艂adnik贸w obok niego opisuje te same bajty i 偶e wystawi艂 go workflow w tym repozytorium, a nie +ten, kto poda艂 Ci plik. + +`--predicate-type` nie jest opcjonalne. Bez niego `gh` szuka atestacji prowenancji builda i +odpowiada `HTTP 404`, bo archiwum, kt贸re pobierasz, jest **podpisywane na maszynie autora**, a nie +budowane na maszynie GitHuba - klucz siedzi na karcie, do kt贸rej 偶aden runner nie si臋ga. +Prowenancja jest atestowana dla niepodpisanego builda wewn膮trz workflow; z wydaniem jedzie spis +sk艂adnik贸w nad podpisanym plikiem. To polecenie pyta GitHuba, jakie atestacje istniej膮. Dow贸d jedzie te偶 **jako plik**, `BeanNetworkTester-vX.Y.Z.sigstore.json`, wi臋c archiwum sprawdzisz wobec dowodu, kt贸ry @@ -1393,10 +1399,12 @@ SBOM jest **podpisany razem z archiwum, kt贸re opisuje**, wi臋c nie da si臋 ich rozdzieli膰: ```bash -gh attestation verify BeanNetworkTester-vX.Y.Z-windows-x64.zip --repo donislawdev/BeanNetworkTester +gh attestation verify BeanNetworkTester-vX.Y.Z-windows-x64.zip --repo donislawdev/BeanNetworkTester --predicate-type https://spdx.dev/Document/v2.3 ``` -Suma kontrolna m贸wi, 偶e plik dotar艂 niezmieniony. Atestacja m贸wi, 偶e **ta** wersja, -z **tym** wykazem sk艂adnik贸w, wysz艂a z workflow wydania tego repozytorium. +Suma kontrolna m贸wi, 偶e plik dotar艂 niezmieniony. Atestacja m贸wi, 偶e **ten** wykaz +sk艂adnik贸w nale偶y do **tego** archiwum i 偶e wystawi艂 go workflow wydania tego +repozytorium. Bez `--predicate-type` polecenie szuka atestacji prowenancji builda +i odpowiada `HTTP 404`. Dokumentacja po angielsku: [README.md](README.md). diff --git a/tests/test_version_and_release.py b/tests/test_version_and_release.py index 52bf275..c87d60a 100644 --- a/tests/test_version_and_release.py +++ b/tests/test_version_and_release.py @@ -897,3 +897,130 @@ def test_the_documented_verify_command_matches_what_we_actually_attest(): if makes_sbom: check(f"{readme}: the predicate type is the SPDX one the workflow makes", "https://spdx.dev/Document/v2.3" in command, f"({command[:160]})") + + # 馃敶 EVERY documented command, not just the one with `--bundle`. Until + # 2026-08-21 this test read the offline line only, and the ONLINE line above + # it - `gh attestation verify -R ` - shipped in 0.5.0 answering + # HTTP 404 for every user, because `gh` defaults to looking for build + # provenance and a hand-signed archive deliberately has none. A guard that + # checks one of two commands is a guard that reports the wrong half is fine. + online = [ln for ln in text.splitlines() if "gh attestation verify" in ln + and "--bundle" not in ln] + check(f"{readme} documents the online verify command", bool(online)) + for command in online: + check(f"{readme}: the online command names a predicate type", + "--predicate-type " in command, + f"(without it gh asks for SLSA provenance and gets 404: {command[:120]})") + if makes_sbom: + check(f"{readme}: the online command asks for the SPDX predicate", + "https://spdx.dev/Document/v2.3" in command, f"({command[:160]})") + + +def test_the_published_release_is_checked_by_a_workflow_not_by_a_person(): + """Something has to run the README's commands against what people download. + + 馃敶 Every other check in the release path looks at an artefact, a draft or a + digest handed between workflows. None of them touches the published release + page, which is the only thing a user ever sees - and that gap is exactly how a + verify command shipped broken twice: once for `v0.5.0-rc.2` (missing `--repo` + and `--predicate-type`) and once for `v0.5.0`, whose online command answered + HTTP 404 because `gh` looks for build provenance a hand-signed archive does not + have. Both were found by a person running the command by hand, afterwards. + + So this pins the workflow to the documentation: the job must run the SAME two + commands the README hands users. If someone rewrites the README's command, this + keeps passing only while the workflow moves with it. + """ + path = os.path.join(ROOT, ".github", "workflows", "verify-release.yml") + check("a workflow verifies the published release", os.path.exists(path), + "(expected .github/workflows/verify-release.yml)") + with open(path, encoding="utf-8") as handle: + body = handle.read() + + check("it runs when a release is published", "types: [published]" in body, + "(a draft is not what users download)") + check("it downloads the published assets", "gh release download" in body) + check("it compares the checksums users are told to compare", + "sha256sum -c SHA256SUMS.txt" in body) + + for flag in ("--bundle", "--repo", "-R ", "--predicate-type", + "https://spdx.dev/Document/v2.3"): + check(f"the workflow runs the documented command with {flag.strip()}", + flag in body, "(it must run the README's command, not an equivalent)") + + # The half a checksum cannot speak for. + check("it checks the Authenticode signature", + "Get-AuthenticodeSignature" in body) + check("it refuses a signature with no timestamp", + "TimeStamperCertificate" in body, + "(without one the signature dies when the certificate expires)") + check("it compares the signing certificate against the pin", + "CODESIGN_SHA256" in body, + "(read out of beantester/legal.py, so the constant has one home)") + check("it writes nothing", "contents: write" not in body, + "(a verifier that can publish is not only a verifier)") + + +def test_release_refuses_a_tag_whose_commit_ci_never_passed(): + """`release.yml` builds and publishes; it does not test. So it has to ASK. + + Step 1 of the release recipe - "tag only a commit CI was green on" - was prose + and nothing enforced it, which the notes said out loud. A red commit plus a tag + published unproven code and the workflow would not have noticed. + + Keyed on the CI workflow by name on purpose: this repository carries a check that + fails for a licensing reason of its own, so "every check is green" would block + every release over something unrelated to the code. + """ + with open(os.path.join(ROOT, ".github", "workflows", "release.yml"), + encoding="utf-8") as handle: + body = handle.read() + check("the release workflow asks whether CI passed", + "actions/runs?head_sha=" in body, + "(it cannot re-run the tests, so it must read their result)") + check("it looks at the CI workflow by name", 'select(.name == "CI")' in body) + check("it has the permission that needs", "actions: read" in body) + check("the gate runs before anything is built", + body.index("actions/runs?head_sha=") < body.index("pyinstaller") + if "pyinstaller" in body.lower() else True, + "(failing after a build wastes the build and reads as a build failure)") + + +def test_the_signing_certificate_expiry_is_a_condition_not_a_note(): + """A date the script prints and draws no conclusion from is decoration. + + The card's certificate expires on a known day. Before 2026-08-21 the first + release after it would have failed inside the signing step, with the card in the + reader - the worst moment to learn about a certificate. The warning also has to + mention the pin, because renewal issues a DIFFERENT certificate and + `legal.CODESIGN_SHA256` has to move with it. + """ + import datetime + import importlib.util + + spec = importlib.util.spec_from_file_location( + "sign_release", os.path.join(ROOT, "tools", "sign_release.py")) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + + now = datetime.datetime(2026, 8, 21, tzinfo=datetime.timezone.utc) + + far = " ".join(module.expiry_notice("2027-08-19T15:26:42+02:00", now)) + check("a certificate with a year left does not shout", "WARNING" not in far, f"({far})") + + soon = " ".join(module.expiry_notice("2026-10-01T10:00:00+02:00", now)) + check("a certificate inside the warning window shouts", "WARNING" in soon, f"({soon})") + check("...and says the pin moves with a renewal", "CODESIGN_SHA256" in soon, f"({soon})") + + for junk in (None, "not-a-date"): + note = " ".join(module.expiry_notice(junk, now)) + check(f"an unreadable expiry ({junk!r}) is reported, not ignored", + "check the card" in note, f"({note})") + + try: + module.expiry_notice("2026-01-01T10:00:00+02:00", now) + check("an EXPIRED certificate refuses to sign", False, + "(it returned a note instead of refusing)") + except SystemExit as exc: + check("the refusal explains the renewal", "CODESIGN_SHA256" in str(exc), + f"({str(exc)[:160]})") diff --git a/tools/sign_release.py b/tools/sign_release.py index f71c645..e7f2404 100644 --- a/tools/sign_release.py +++ b/tools/sign_release.py @@ -27,18 +27,25 @@ accident this catches; 5. repacks the archive, writes ``SHA256SUMS.txt`` over what it just made; 6. uploads both to the DRAFT release and asks the workflow to attest the signed - bytes, so the ``.sigstore.json`` a user verifies describes the file they hold. + bytes, so the ``.sigstore.json`` a user verifies describes the file they hold; +7. **waits for that attestation and confirms the draft is complete** - four assets, + the published checksums naming the digest that was actually signed, and the + release still a draft. Until this existed the script ended at "dispatched, go + look", and a draft missing one file looks almost exactly like a finished one. Nothing here publishes. The release stays a draft until a person looks at it and -presses the button. +presses the button - and pressing it runs ``verify-release.yml``, which downloads +the published assets and re-checks them the way a user would. """ import argparse +import datetime import hashlib import json import os import shutil import subprocess import sys +import time import zipfile ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) @@ -89,6 +96,131 @@ def find_signtool(): return found[-1] +WARN_DAYS = 90 + + +def _now(): + return datetime.datetime.now(datetime.timezone.utc) + + +def expiry_notice(not_after, now): + """Lines to print about the certificate's remaining life - and a refusal if none. + + 馃敶 The card's certificate expires on a known date (2027-08-19 for the current + one), and until 2026-08-21 the script PRINTED that date and drew no conclusion + from it. The first release after expiry would therefore have failed in the middle + of the ritual, at the signing step, with the card already in the reader - the + worst possible moment to discover a certificate problem. + + Renewal issues a NEW certificate, so `legal.CODESIGN_SHA256` moves with it. The + warning says so, because a session that renews the certificate and does not know + that will next meet the pin as a refusal it cannot explain. + + Pure, so it can be checked without a card: `now` is passed in. + """ + if not not_after: + return [" 馃敶 the store did not report an expiry date - check the card by hand"] + try: + when = datetime.datetime.fromisoformat(str(not_after)) + except ValueError: + return [" 馃敶 could not read the expiry date %r - check the card by hand" + % (not_after,)] + if when.tzinfo is None: + when = when.replace(tzinfo=datetime.timezone.utc) + days = (when - now).days + if days < 0: + raise SystemExit( + "sign_release: the pinned certificate EXPIRED %d days ago (%s).\n" + "Signing with it now produces a signature Windows will reject. Renew the\n" + "certificate, then move legal.CODESIGN_SHA256 to the new one's SHA-256 -\n" + "a renewal is a different certificate, not the same one with a new date." + % (-days, when.date())) + if days <= WARN_DAYS: + return [" 馃敶 WARNING: %d days left on this certificate (%s)." % (days, when.date()), + " Renewing issues a NEW certificate, so legal.CODESIGN_SHA256 has to", + " move with it or the next release refuses to sign at all."] + return [" %d days left on the certificate" % days] + + +EXPECTED_ASSETS = (".zip", "SHA256SUMS.txt", ".spdx.json", ".sigstore.json") + + +def _gh_json(*args): + """gh, parsed, without ending the ritual on failure - for polling.""" + result = subprocess.run(["gh"] + list(args), text=True, capture_output=True) + if result.returncode != 0: + return None + try: + return json.loads(result.stdout or "null") + except ValueError: + return None + + +def confirm_draft(tag, digest, wait_seconds): + """Is the draft actually complete and describing the bytes we just signed? + + 馃敶 This step exists because the script used to end at "dispatched, go look". The + upload and the dispatch are two calls, phase C is a third thing entirely, and a + half-finished draft looks almost exactly like a finished one - three assets + instead of four. On 2026-08-20 that difference (six rows in the web UI instead of + seven) took a person and a session several minutes to explain, on a release that + was in fact fine. On a release that was NOT fine it would have been published. + + Waits for phase C rather than assuming it: attesting takes well under a minute, + but "well under a minute" is not "already done" at the moment the dispatch + returns. + + Returns (ok, lines_to_print) and never raises, so the caller decides. + """ + deadline = time.monotonic() + max(0, wait_seconds) + lines, assets = [], [] + while True: + data = _gh_json("release", "view", tag, "--repo", REPO, + "--json", "assets,isDraft") + assets = [a.get("name", "") for a in (data or {}).get("assets", [])] + missing = [kind for kind in EXPECTED_ASSETS + if not any(name.endswith(kind) for name in assets)] + if not missing: + break + if time.monotonic() >= deadline: + lines.append(" waited %ds and the draft is still missing: %s" + % (wait_seconds, ", ".join(missing))) + lines.append(" assets present: %s" % (", ".join(sorted(assets)) or "none")) + lines.append(" '%s' attaches the .sigstore.json - check its run log." + % ATTEST_WORKFLOW) + return False, lines + time.sleep(5) + + for name in sorted(assets): + lines.append(" asset %s" % name) + + # The digest is checked against what the RELEASE carries, not against the local + # file we made - those are the same bytes only if the upload really landed. + where = os.path.join(ROOT, "build", "signing", tag, "confirm") + shutil.rmtree(where, ignore_errors=True) + os.makedirs(where, exist_ok=True) + got = subprocess.run(["gh", "release", "download", tag, "--repo", REPO, + "--pattern", "SHA256SUMS.txt", "--dir", where], + text=True, capture_output=True) + if got.returncode != 0: + lines.append(" could not download SHA256SUMS.txt back from the draft") + return False, lines + with open(os.path.join(where, "SHA256SUMS.txt"), encoding="utf-8") as handle: + published = handle.read() + if digest not in published: + lines.append(" 馃敶 SHA256SUMS.txt on the release does NOT name the digest we signed") + lines.append(" signed: %s" % digest) + lines.append(" published: %s" % published.strip()) + return False, lines + lines.append(" checksums on the release name the digest we signed (%s...)" % digest[:16]) + if (_gh_json("release", "view", tag, "--repo", REPO, "--json", "isDraft") or + {}).get("isDraft") is False: + lines.append(" 馃敶 this release is NOT a draft any more - it is already public") + return False, lines + lines.append(" PASS: four assets, digests agree, still a draft") + return True, lines + + def signing_thumbprint(): """The SHA-1 thumbprint of the certificate whose DER bytes hash to our pin. @@ -116,6 +248,8 @@ def signing_thumbprint(): if entry.get("sha256") == CODESIGN_SHA256: print(" certificate: %s" % entry.get("subject", "").split(",")[0]) print(" expires: %s" % entry.get("notAfter")) + for line in expiry_notice(entry.get("notAfter"), now=_now()): + print(line) return entry["thumb"] raise SystemExit( "sign_release: the pinned certificate (%s...) is not in the Windows store. " @@ -151,6 +285,9 @@ def main(argv=None): help="do everything except sign, upload and dispatch") parser.add_argument("--work", default=os.path.join(ROOT, "build", "signing"), help="scratch directory (default: build/signing)") + parser.add_argument("--wait", type=int, default=300, metavar="SECONDS", + help="how long to wait for the attestation workflow to " + "finish attaching its bundle (default: 300)") args = parser.parse_args(argv) if os.name != "nt": @@ -162,7 +299,7 @@ def main(argv=None): os.makedirs(work) print("working in %s" % work) - print("\n[1/6] fetching the build this tag produced") + print("\n[1/7] fetching the build this tag produced") run(["gh", "run", "download", "--repo", REPO, "--name", "unsigned-build-%s" % args.tag, "--dir", work]) archives = [f for f in os.listdir(work) if f.endswith(".zip")] @@ -171,10 +308,10 @@ def main(argv=None): % archives) archive = os.path.join(work, archives[0]) - print("\n[2/6] verifying what the workflow says it built") + print("\n[2/7] verifying what the workflow says it built") run(["gh", "attestation", "verify", archive, "--repo", REPO]) - print("\n[3/6] unpacking") + print("\n[3/7] unpacking") unpacked = os.path.join(work, "unpacked") with zipfile.ZipFile(archive) as zf: zf.extractall(unpacked) @@ -187,7 +324,7 @@ def main(argv=None): exe = exes[0] print(" %s (%d bytes, unsigned)" % (os.path.basename(exe), os.path.getsize(exe))) - print("\n[4/6] signing with the card") + print("\n[4/7] signing with the card") thumbprint = signing_thumbprint() signtool = find_signtool() command = [signtool, "sign", "/sha1", thumbprint, "/fd", "sha256", @@ -205,7 +342,7 @@ def main(argv=None): % (CODESIGN_SHA256, actual)) print(" signed by the pinned certificate, timestamped") - print("\n[5/6] repacking and checksumming") + print("\n[5/7] repacking and checksumming") signed = os.path.join(work, archives[0]) if not args.dry_run: os.remove(archive) @@ -219,7 +356,7 @@ def main(argv=None): handle.write("%s *%s\n" % (sha256_of(signed), archives[0])) print(" %s" % sha256_of(signed)) - print("\n[6/6] handing it back to the workflow") + print("\n[6/7] handing it back to the workflow") if args.dry_run: print(" DRY RUN, would upload the archive and SHA256SUMS.txt to the draft") print(" DRY RUN, would dispatch %s with the digest" % ATTEST_WORKFLOW) @@ -229,9 +366,19 @@ def main(argv=None): "--repo", REPO, "--clobber"]) run(["gh", "workflow", "run", ATTEST_WORKFLOW, "--repo", REPO, "-f", "tag=%s" % args.tag, "-f", "digest=%s" % sha256_of(signed)]) + + print("\n[7/7] confirming the draft is complete") + ok, lines = confirm_draft(args.tag, sha256_of(signed), args.wait) + for line in lines: + print(line) + if not ok: + raise SystemExit( + "sign_release: the draft is NOT complete. Nothing was published, so " + "nothing is broken - but do not press publish until the missing piece " + "is there. Re-running this script is safe (the upload uses --clobber).") print("\nDone. The release is still a DRAFT.") - print("Wait for '%s' to attach the .sigstore.json, read the draft, then publish it." - % ATTEST_WORKFLOW) + print("Read the draft, then publish it. Publishing runs 'Verify the published " + "release', which re-checks the bytes the way a user would.") return 0 From 84c5a3ffa814d5cc086bc2677117aed7fe8bdde3 Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Fri, 21 Aug 2026 13:26:38 +0200 Subject: [PATCH 2/2] fix(docs): keep the README guards happy about the corrected verify command Two guards caught the previous commit, and both were right. The mutation registry pins the documented verify command by removing its predicate type and expecting a test to redden. Its pattern was the flag on its own, which matched exactly once while only one command carried it. Adding the flag to the other two occurrences made that pattern match three places, and a pattern that matches three places proves nothing about any of them. There are two entries now, anchored on `.sigstore.json` and on `-R` - each unique, each version-independent, one for the offline command and one for the online one that actually broke. Both were run for real and both are caught. The other is the no-semicolons rule for README prose: the sentence explaining why `--predicate-type` is required used one. It is two sentences now, in both languages. Co-authored-by: Claude Opus 5 --- README.md | 4 ++-- README.pl.md | 2 +- tests/test_mutation_registry.py | 25 ++++++++++++++++++++++--- 3 files changed, 25 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index d157b04..1dbdf5e 100644 --- a/README.md +++ b/README.md @@ -1510,8 +1510,8 @@ repository rather than by whoever handed you the file. `--predicate-type` is not optional. Without it `gh` looks for a build-provenance statement and answers `HTTP 404`, because the archive you download was **signed on the maintainer's machine**, not built on a runner - the signing key is on a card that no runner can reach. Provenance is -attested for the unsigned build inside the workflow; what travels with the release is the bill of -materials over the signed file. +attested for the unsigned build inside the workflow. What travels with the release is the bill +of materials over the signed file. That command asks GitHub which attestations exist. The proof also ships **as a file**, `BeanNetworkTester-vX.Y.Z.sigstore.json`, so you can check the archive against evidence that diff --git a/README.pl.md b/README.pl.md index 1b11dd0..b73cad1 100644 --- a/README.pl.md +++ b/README.pl.md @@ -1365,7 +1365,7 @@ ten, kto poda艂 Ci plik. `--predicate-type` nie jest opcjonalne. Bez niego `gh` szuka atestacji prowenancji builda i odpowiada `HTTP 404`, bo archiwum, kt贸re pobierasz, jest **podpisywane na maszynie autora**, a nie budowane na maszynie GitHuba - klucz siedzi na karcie, do kt贸rej 偶aden runner nie si臋ga. -Prowenancja jest atestowana dla niepodpisanego builda wewn膮trz workflow; z wydaniem jedzie spis +Prowenancja jest atestowana dla niepodpisanego builda wewn膮trz workflow. Z wydaniem jedzie spis sk艂adnik贸w nad podpisanym plikiem. To polecenie pyta GitHuba, jakie atestacje istniej膮. Dow贸d jedzie te偶 **jako plik**, diff --git a/tests/test_mutation_registry.py b/tests/test_mutation_registry.py index 830fa38..35314cf 100644 --- a/tests/test_mutation_registry.py +++ b/tests/test_mutation_registry.py @@ -1051,10 +1051,29 @@ # The command in the README stops matching what we attest, and every user who # follows it gets an error. Nothing here runs `gh`, so only this pairing can # notice. - "label": "release: the documented verify command loses its predicate type", + # + # 馃敶 Anchored on `.sigstore.json` rather than on the flag alone. The flag now + # appears three times in README.md - it was added to the two ONLINE forms on + # 2026-08-21, after the one without it shipped in 0.5.0 answering HTTP 404 - + # and a pattern that matches three places proves nothing about any of them. + # This anchor is version-independent: the file name carries the version, the + # extension does not. + "label": "release: the documented OFFLINE verify command loses its predicate type", "file": "README.md", - "old": " --predicate-type https://spdx.dev/Document/v2.3", - "new": "", + "old": ".sigstore.json --repo donislawdev/BeanNetworkTester" + " --predicate-type https://spdx.dev/Document/v2.3", + "new": ".sigstore.json --repo donislawdev/BeanNetworkTester", + "test": "test_the_documented_verify_command_matches_what_we_actually_attest", + }, + { + # The other half, and the one that actually broke. `-R` is the online form's + # short flag and appears nowhere else in the file, so this anchor stays unique + # for the same reason the one above does. + "label": "release: the documented ONLINE verify command loses its predicate type", + "file": "README.md", + "old": "-R donislawdev/BeanNetworkTester" + " --predicate-type https://spdx.dev/Document/v2.3", + "new": "-R donislawdev/BeanNetworkTester", "test": "test_the_documented_verify_command_matches_what_we_actually_attest", }, {