fix(ci): re-run hygiene check on every bump-version push retry (#678)

.github/workflows/ci.yml

@@ -110,6 +110,11 @@ jobs:
           persist-credentials: false
       - name: Test publish/backfill scripts
         run: ./scripts/test_publish_spaces.sh
+      # Same reasoning for the version-bump push: it only ever runs on pushes to
+      # main, writing directly to main, so its retry path is untestable in PR CI
+      # unless it is exercised here against a real local remote.
+      - name: Test version-bump push script
+        run: ./scripts/test_push_version_bump.sh
 
   clippy-and-test:
     name: Build, Test, Clippy
@@ -701,29 +706,13 @@ jobs:
       - name: Static hygiene and version check before push
         run: python scripts/check_repo_hygiene.py --mode static
 
+      # The retry loop lives in a script so it can be tested (see
+      # scripts/test_push_version_bump.sh, run by the `release-scripts` job).
+      # Each retry re-bumps onto the new tip and re-runs the hygiene check
+      # above against that new tree before pushing it.
       - name: Push changes
         if: success()
-        run: |
-          set -eu
-          BRANCH="${{ github.ref_name }}"
-          # The bump commit must fast-forward main. When another commit lands on
-          # main during this run (concurrent merges), the first push is rejected
-          # as non-fast-forward. Re-base the bump onto the new tip and retry.
-          # Because two concurrent runs can compute the same next version, we
-          # reset to the fresh tip and re-run the bump so the version is derived
-          # from the true current version instead of replaying a stale bump.
-          for attempt in 1 2 3 4 5; do
-            if git push origin HEAD:"$BRANCH"; then
-              echo "Pushed version bump on attempt $attempt"
-              exit 0
-            fi
-            echo "Push rejected on attempt $attempt (main advanced); re-bumping from latest tip..."
-            git fetch origin "$BRANCH"
-            git reset --hard "origin/$BRANCH"
-            python scripts/bump_version.py --update-all
-          done
-          echo "::error::Failed to push version bump after 5 attempts (main kept advancing)."
-          exit 1
+        run: ./scripts/push_version_bump.sh "${{ github.ref_name }}"
 
       - name: Tag the new version
         if: success()

History/dev-diary/2026/2026-08-14-issue-678-bump-version-hygiene-on-retry.md

@@ -0,0 +1,101 @@
+# 2026-08-14 — `bump-version` pushed to `main` unchecked on any push retry (#678)
+
+## Symptom
+
+None observed in production — and that is the point. The gap needs a genuine
+race to surface: another commit must land on `main` between the `bump-version`
+job's checkout and its push. Rare enough to go unnoticed, and the place it
+surfaces is the branch the check exists to protect.
+
+Split out of the #674 review, where CodeRabbit raised it. Pre-existing; #674 did
+not touch it.
+
+## Root cause
+
+`.github/workflows/ci.yml` ran the hygiene gate exactly once, as its own step,
+with a comment stating the requirement plainly:
+
+```yaml
+# This job writes directly to main, so it must prove hygiene and version
+# agreement itself, immediately before pushing (REPOSITORY_HYGIENE.md §8).
+- name: Static hygiene and version check before push
+  run: python scripts/check_repo_hygiene.py --mode static
+```
+
+The `Push changes` step then retried up to five times, and each retry rebuilt
+the commit from scratch:
+
+```bash
+git fetch origin "$BRANCH"
+git reset --hard "origin/$BRANCH"
+python scripts/bump_version.py --update-all
+```
+
+That is a **new commit on a new tree** — the concurrent merge's content plus a
+fresh bump — and it went out without the gate running again. The step's own
+comment was true only of the first attempt. A concurrent merge carrying a
+hygiene violation, or a bump that drifted against the new tree, reached `main`
+unchecked and was caught only by the *next* run, after it had landed.
+
+## The fix
+
+The retry loop moved to `scripts/push_version_bump.sh`, which re-runs the
+hygiene check on the re-bumped tree before each subsequent push. A violation
+**aborts**:
+
+```bash
+if ! $HYGIENE_CMD; then
+  echo "::error::Static hygiene check failed on the re-bumped tree (attempt $attempt); refusing to push."
+  exit 1
+fi
+```
+
+Aborting, not skipping to the next attempt — retrying past a violation would
+defeat the gate. Aborting leaves `main` un-bumped, which the next push to `main`
+recovers. The non-retry path is unchanged: still exactly one check, in the
+workflow step, before the first attempt.
+
+## Why it moved out of the YAML
+
+Inline workflow shell cannot be tested — it only ever executes in production, on
+pushes to `main`, on the rare retry path. That is the same hazard
+`scripts/test_publish_spaces.sh` exists to close for the release scripts, and it
+gets the same treatment here.
+
+`scripts/test_push_version_bump.sh` runs the real script against a **real git
+repository and a real bare remote**, with genuine non-fast-forward rejections
+produced by a second clone committing to the remote mid-flight. Only the two
+shelled-out commands are stubbed — `BUMP_CMD` and `HYGIENE_CMD`, as recording
+fakes that log the HEAD each one saw — because the production versions mutate
+the working repo and run Cargo. `git push` is never stubbed; the assertions are
+made against where the remote tip actually moved.
+
+## Risk class
+
+**R3** — release controls (root `testing.md` §5), which requires negative and
+failure-path coverage. The failure-path case asserts the *absence* of the write:
+when hygiene fails on a retry, the script exits non-zero **and** the remote tip
+is byte-identical to where it started.
+
+## Red evidence
+
+Against the faithful extraction of the buggy loop, 7 of 19 assertions failed.
+The one that matters:
+
+```text
+FAIL nothing was pushed - the remote tip is exactly where it was
+       expected: 0e721debb32938a8cf5da424cc1b7bb75d3f0f02
+       actual:   b169625d9ecf277cb4acd3d316750132847f2a32
+```
+
+The old loop pushed `b169625` — the re-bumped tree carrying the concurrent
+merge — to the remote despite hygiene rejecting it. That is the defect, observed
+rather than argued.
+
+Red: `6bab170`. Green: `e60b8c9`. 19/19 after the fix.
+
+## Coverage added
+
+`Release Script Tests` now runs `scripts/test_push_version_bump.sh` alongside
+`scripts/test_publish_spaces.sh`, so the retry path is exercised on every PR
+rather than only during a real race on `main`.

scripts/push_version_bump.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Push the post-merge version-bump commit to a branch, re-bumping on rejection.
+#
+# Extracted from the `bump-version` job in .github/workflows/ci.yml so the retry
+# behaviour can be tested (inline workflow shell cannot be).
+#
+# Usage: scripts/push_version_bump.sh <branch>
+#
+# The two commands this shells out to are overridable so the tests can record
+# and control them; the defaults are what CI runs:
+#   BUMP_CMD     - re-derives the next version from the fresh tip and commits it
+#   HYGIENE_CMD  - proves the tree about to be pushed is hygienic
+set -eu
+
+BRANCH="${1:-${GITHUB_REF_NAME:-}}"
+[ -n "$BRANCH" ] || { echo "usage: $0 <branch>" >&2; exit 64; }
+
+BUMP_CMD="${BUMP_CMD:-python scripts/bump_version.py --update-all}"
+HYGIENE_CMD="${HYGIENE_CMD:-python scripts/check_repo_hygiene.py --mode static}"
+
+# The bump commit must fast-forward main. When another commit lands on main
+# during this run (concurrent merges), the first push is rejected as
+# non-fast-forward. Re-base the bump onto the new tip and retry. Because two
+# concurrent runs can compute the same next version, we reset to the fresh tip
+# and re-run the bump so the version is derived from the true current version
+# instead of replaying a stale bump.
+ATTEMPTS=5
+for attempt in 1 2 3 4 5; do
+  if git push origin HEAD:"$BRANCH"; then
+    echo "Pushed version bump on attempt $attempt"
+    exit 0
+  fi
+  # After the final rejection there is no further push, so re-bumping would
+  # build a commit nothing will ever send - and the real bump shells out to
+  # Cargo, so that is not free. Give up here instead.
+  if [ "$attempt" -eq "$ATTEMPTS" ]; then
+    break
+  fi
+  echo "Push rejected on attempt $attempt (main advanced); re-bumping from latest tip..."
+  git fetch origin "$BRANCH"
+  git reset --hard "origin/$BRANCH"
+  # shellcheck disable=SC2086  # intentional word splitting: command + args
+  $BUMP_CMD
+  # The re-bump produced a new commit on a new tree - whatever landed on
+  # `$BRANCH` concurrently, plus a fresh bump. The check the job ran before the
+  # first attempt says nothing about that tree, so hygiene and version agreement
+  # must be re-proven here, immediately before this push
+  # (REPOSITORY_HYGIENE.md §8).
+  # shellcheck disable=SC2086  # intentional word splitting: command + args
+  if ! $HYGIENE_CMD; then
+    # Abort rather than retry: retrying past a violation would defeat the gate.
+    # This leaves the branch un-bumped, which the next push to it recovers.
+    echo "::error::Static hygiene check failed on the re-bumped tree (attempt $attempt); refusing to push."
+    exit 1
+  fi
+done
+echo "::error::Failed to push version bump after 5 attempts (main kept advancing)."
+exit 1