From e13d160e666f9c302158b5745b22cb703f76036d Mon Sep 17 00:00:00 2001 From: logbie Date: Wed, 19 Aug 2026 04:42:13 -0500 Subject: [PATCH] fix(ci): package the Linux tarball reproducibly so a nightly can be re-run `scripts/publish_spaces.sh` documents versioned keys as immutable with a deliberate retry path: identical bytes are a no-op, different bytes abort the publish, "so a retry after a partial failure completes rather than trips over the objects the previous attempt already landed." The Linux tarball could never take that path. Packaging the same commit twice produced different bytes in three independent ways, none of them the compiled output: - BUILD_INFO's `built:` recorded `date -u`, the wall clock. - `tar czf` stored each member's mtime, i.e. when `cp` ran. - Member order followed directory-read order and reshuffled per run. So the guard fired on a rebuild of an already-published version+sha and aborted the whole publish, taking the tag and the GitHub release with it (they are later steps in the same job). Re-running a nightly - the standard remediation, and the documented way to verify a nightly-only change - was structurally impossible once that version had published. Evidence: nightly run 32235610626 (manual dispatch on main @36de4fa7, v26.8.8) built green on both Windows and Linux, then failed at "Publish artifacts to DigitalOcean Spaces" against the 2026-08-15 publish of the same commit. Comparing the two tarballs, `wfl` and `wfl-lsp` are bit-identical (d8838658..., 7782faab...); the sole content difference in the entire archive is BUILD_INFO's timestamp line. The build is already reproducible - only the packaging was not. Fix: derive a SOURCE_DATE_EPOCH from the commit's committer date and use it for both BUILD_INFO's `built:` and tar's `--mtime`, and add `--sort=name --owner=0 --group=0 --numeric-owner`. gzip already records MTIME=0 because tar -z compresses a pipe. `built:` now means the commit's date rather than the moment the runner happened to package it. That is the one judgement call here: it is the field that has to become commit-derived for the artifact to be stable, and the sha and version already identify the build uniquely. Docs updated to match. Verified locally by packaging the identical binaries twice, seconds apart, with each recipe: the current one produced ead51b15... then fc9773cb...; the new one produced 3e18d835... both times. Risk class R0 (CI mechanics; no runtime behaviour changes). actionlint clean on nightly.yml. --- .github/workflows/nightly.yml | 26 +++++++++++++++++++++++-- Docs/02-getting-started/installation.md | 2 +- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 68fea4b7..230e0e81 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -550,17 +550,39 @@ jobs: cp "target/$TARGET/release/wfl-lsp" "dist/$DIR/wfl-lsp" strip "dist/$DIR/wfl" "dist/$DIR/wfl-lsp" cp README.md LICENSE "dist/$DIR/" + + # publish_spaces.sh treats versioned keys as immutable: identical bytes + # are a no-op and different bytes abort the publish, which is what lets + # a publish that half-landed be repaired by re-running the nightly. + # That contract only holds if the same commit packages to the same + # bytes, so everything below that would otherwise vary run to run is + # pinned to the commit rather than to the wall clock: + # + # - BUILD_INFO's `built:` is the commit's own committer date. It is + # still the moment this artifact corresponds to, and unlike + # `date -u` it does not change when the same commit is rebuilt. + # - --sort=name fixes member order, which otherwise follows + # directory-read order and reshuffles between runs. + # - --mtime pins the header timestamps, which otherwise record when + # `cp` happened. + # - --owner/--group/--numeric-owner drop the runner's uid/gid names. + # + # gzip already records MTIME=0 here because tar -z compresses a pipe. + SOURCE_DATE_EPOCH="$(git show -s --format=%ct HEAD)" cat > "dist/$DIR/BUILD_INFO" <-linux-x86_64/` directory containing: - `wfl` - the WFL compiler and runtime - `wfl-lsp` - the Language Server, for editor integration - `README.md`, `LICENSE` -- `BUILD_INFO` - version, commit, build time, and target triple +- `BUILD_INFO` - version, commit, commit date, and target triple ### Step 3: Install