(MOT-4299) feat: stage worker releases before promotion - #692
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughThe release pipeline now creates staged candidates, validates release evidence and smoke-test gates, publishes candidates to ChangesStaged worker release
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant ReleaseWorkflow
participant CandidateSmoke
participant CandidateEvidence
participant PromoteWorker
participant RegistryAPI
ReleaseWorkflow->>CandidateSmoke: install and test worker@next
CandidateSmoke-->>ReleaseWorkflow: return smoke artifacts
ReleaseWorkflow->>CandidateEvidence: build and validate release-candidate.json
CandidateEvidence-->>ReleaseWorkflow: return candidate_ready
PromoteWorker->>CandidateEvidence: validate immutable candidate evidence
PromoteWorker->>RegistryAPI: promote verified version to latest
RegistryAPI-->>PromoteWorker: return promotion metadata
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
skill-check — worker0 verified, 53 skipped (no docs/).
Four for four. Nicely done. |
ytallo
marked this pull request as ready for review
August 4, 2026 16:26
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/registry_release.py:
- Around line 76-85: The promotion flow around promotion_payload() must not
require ownership of next when current_latest already equals version. Return a
verified no-op for this idempotent retry path, or use a promotion API that omits
expected_tag: next, while preserving the existing next precondition for first
promotions. Update the related tests to stop expecting expected_tag: next on
idempotent retries.
In @.github/scripts/release_candidate.py:
- Around line 92-96: Update the Harness validation flow around the
evidence.get("harness_gate_required") check to derive the expected gate
requirement from trusted promotion inputs rather than trusting the candidate
evidence. Compare the derived policy with the evidence before validating
harness_quickstart and harness_e2e results, and ensure a harness worker with
harness_gate_required=False is rejected when the trusted policy requires the
gates; add coverage for that case.
In `@docs/sops/release.md`:
- Around line 57-77: Update the release flow description around the
create-release stage and its “Public GitHub prerelease” output to distinguish
release types: describe staged Registry releases and semantic prereleases as
GitHub prereleases, while stable workers with interface_smoke disabled are
documented as normal GitHub Releases. Keep the existing workflow stages and
outputs unchanged apart from this release-type clarification.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 2c3d52bd-f01c-4bf8-b122-1ff1f90da057
📒 Files selected for processing (19)
.github/scripts/parse_release_tag.py.github/scripts/registry_release.py.github/scripts/release_candidate.py.github/scripts/tests/test_parse_release_tag.py.github/scripts/tests/test_registry_release.py.github/scripts/tests/test_release_candidate.py.github/workflows/_candidate-smoke.yml.github/workflows/_container.yml.github/workflows/_harness-e2e.yml.github/workflows/_publish-registry.yml.github/workflows/create-tag.yml.github/workflows/harness-e2e-deployed.yml.github/workflows/harness-quickstart.yml.github/workflows/promote-worker.yml.github/workflows/release.ymldocs/sops/release.mdharness/tests/e2e/run-deployed-ci.shharness/tests/quickstart/README.mdharness/tests/quickstart/run-ci.sh
Comment on lines
+76
to
+85
| # A first promotion must still own `next`. Once Registry latest already | ||
| # points at the requested immutable version, allow an idempotent retry to | ||
| # repair GitHub/GHCR/Slack even if a newer candidate has moved `next`. | ||
| if current_latest != version and current_next != version: | ||
| raise RegistryError(f"next points to {current_next}, expected {version}") | ||
| encoded_worker = urllib.parse.quote(worker, safe="") | ||
| status, response = request_json( | ||
| "PUT", | ||
| f"{api_url.rstrip('/')}/w/{encoded_worker}/tags/latest", | ||
| promotion_payload(version, current_latest), |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 8 \
'expected_tag|expected_current_version|tags/latest|promotion_payload' \
.Repository: iii-hq/workers
Length of output: 152
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files matching registry_release.py:"
fd -a 'registry_release\.py$' . || true
echo
echo "Changed files/stat:"
git diff --stat || true
echo
echo "Search target filename:"
if [ -f .github/scripts/registry_release.py ]; then
nl -ba .github/scripts/registry_release.py | sed -n '1,180p'
else
echo ".github/scripts/registry_release.py not found"
fi
echo
echo "Broader search for target terms (case-insensitive):"
rg -n -i -C 6 \
'expected_tag|expected_current_version|tags/latest|promotion_payload|current_next|current_latest' .github src . 2>/dev/null || trueRepository: iii-hq/workers
Length of output: 275
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "pwd:"
pwd
echo
echo "Files matching registry_release.py:"
find . -type f -name 'registry_release.py' 2>/dev/null || true
echo
echo "Changed files/stat:"
git diff --stat 2>/dev/null || true
echo
echo "File contents:"
if [ -f .github/scripts/registry_release.py ]; then
awk '{ printf "%6d\t%s\n", NR, $0 }' .github/scripts/registry_release.py | sed -n '1,180p'
else
echo ".github/scripts/registry_release.py not found"
fi
echo
echo "Broader search for target terms:"
grep -RIn -C 6 -E 'expected_tag|expected_current_version|tags/latest|promotion_payload|current_next|current_latest' .github src . 2>/dev/null || true
echo
echo "Git status short:"
git status --short 2>/dev/null || trueRepository: iii-hq/workers
Length of output: 17601
Remove the next precondition from the idempotent retry path.
When latest already equals version, latest has moved, and next is version, the check allows the retry, but promotion_payload() still sends expected_tag: next. A Registry that requires ownership of the source tag can reject this retry. Return a verified no-op when current_latest == version, or use a promotion API that does not require ownership of next. Update the tests to stop asserting that idempotent reties include expected_tag: next.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/scripts/registry_release.py around lines 76 - 85, The promotion flow
around promotion_payload() must not require ownership of next when
current_latest already equals version. Return a verified no-op for this
idempotent retry path, or use a promotion API that omits expected_tag: next,
while preserving the existing next precondition for first promotions. Update the
related tests to stop expecting expected_tag: next on idempotent retries.
Comment on lines
+92
to
+96
| if evidence.get("harness_gate_required"): | ||
| if results.get("harness_quickstart") != "success": | ||
| failures.append("Harness quickstart gate did not succeed") | ||
| if results.get("harness_e2e") != "success": | ||
| failures.append("Harness E2E gate did not succeed") |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Derive the Harness gate policy independently of the evidence.
The evidence controls harness_gate_required. If a harness candidate records this field as false, validation accepts skipped Harness gates while candidate_ready remains true.
Require the expected gate policy from trusted promotion inputs. Compare that policy with the evidence before checking the results. Add a test for worker="harness" with harness_gate_required=False.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/scripts/release_candidate.py around lines 92 - 96, Update the
Harness validation flow around the evidence.get("harness_gate_required") check
to derive the expected gate requirement from trusted promotion inputs rather
than trusting the candidate evidence. Compare the derived policy with the
evidence before validating harness_quickstart and harness_e2e results, and
ensure a harness worker with harness_gate_required=False is rejected when the
trusted policy requires the gates; add coverage for that case.
Comment on lines
+57
to
+77
| setupJob --> ghRelease[create public GitHub prerelease] | ||
| ghRelease --> buildBinary["binary: _rust-binary.yml"] | ||
| ghRelease --> buildImage["image: _container.yml"] | ||
| ghRelease --> buildBundle["bundle: _bundle.yml"] | ||
| buildBinary --> publishJob[_publish-registry.yml] | ||
| buildImage --> publishJob | ||
| buildBundle --> publishJob | ||
| publishJob --> postPublish["POST /publish + skills"] | ||
| publishJob --> candidateSmoke[resolve / install / boot next] | ||
| candidateSmoke --> harnessGate{Harness or dependency?} | ||
| harnessGate -->|yes| quickstart[Harness quickstart] | ||
| quickstart --> e2e[Harness deployed E2E] | ||
| harnessGate -->|no| evidence[candidate evidence] | ||
| e2e --> evidence | ||
| evidence --> promotion[manual Promote Worker] | ||
| promotion --> latest[Registry latest + GitHub Release] | ||
| ``` | ||
|
|
||
| | Stage | Job | Output | | ||
| |---|---|---| | ||
| | setup | Parse tag + `iii.worker.yaml`; detect web bundle / smoke opt-out | worker, version, deploy, targets, … | | ||
| | create-release | GitHub Release shell | Release page for the tag | | ||
| | create-release | Public GitHub prerelease, never repository-global Latest | Release page and downloadable assets | |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Correct the release-type description.
Line 57 and Line 77 state that every release creates a GitHub prerelease. A stable worker with interface_smoke: false has staged=false, so .github/workflows/release.yml creates a normal GitHub Release when is_prerelease=false. Restrict the prerelease description to staged Registry releases and semantic prereleases.
Proposed documentation update
- setupJob --> ghRelease[create public GitHub prerelease]
+ setupJob --> ghRelease[create GitHub Release]
...
-| create-release | Public GitHub prerelease, never repository-global Latest | Release page and downloadable assets |
+| create-release | Staged Registry releases and semantic prereleases: public GitHub prerelease. Stable GitHub-only releases: normal GitHub Release. Never repository-global Latest. | Release page and downloadable assets |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| setupJob --> ghRelease[create public GitHub prerelease] | |
| ghRelease --> buildBinary["binary: _rust-binary.yml"] | |
| ghRelease --> buildImage["image: _container.yml"] | |
| ghRelease --> buildBundle["bundle: _bundle.yml"] | |
| buildBinary --> publishJob[_publish-registry.yml] | |
| buildImage --> publishJob | |
| buildBundle --> publishJob | |
| publishJob --> postPublish["POST /publish + skills"] | |
| publishJob --> candidateSmoke[resolve / install / boot next] | |
| candidateSmoke --> harnessGate{Harness or dependency?} | |
| harnessGate -->|yes| quickstart[Harness quickstart] | |
| quickstart --> e2e[Harness deployed E2E] | |
| harnessGate -->|no| evidence[candidate evidence] | |
| e2e --> evidence | |
| evidence --> promotion[manual Promote Worker] | |
| promotion --> latest[Registry latest + GitHub Release] | |
| ``` | |
| | Stage | Job | Output | | |
| |---|---|---| | |
| | setup | Parse tag + `iii.worker.yaml`; detect web bundle / smoke opt-out | worker, version, deploy, targets, … | | |
| | create-release | GitHub Release shell | Release page for the tag | | |
| | create-release | Public GitHub prerelease, never repository-global Latest | Release page and downloadable assets | | |
| setupJob --> ghRelease[create GitHub Release] | |
| ghRelease --> buildBinary["binary: _rust-binary.yml"] | |
| ghRelease --> buildImage["image: _container.yml"] | |
| ghRelease --> buildBundle["bundle: _bundle.yml"] | |
| buildBinary --> publishJob[_publish-registry.yml] | |
| buildImage --> publishJob | |
| buildBundle --> publishJob | |
| publishJob --> candidateSmoke[resolve / install / boot next] | |
| candidateSmoke --> harnessGate{Harness or dependency?} | |
| harnessGate -->|yes| quickstart[Harness quickstart] | |
| quickstart --> e2e[Harness deployed E2E] | |
| harnessGate -->|no| evidence[candidate evidence] | |
| e2e --> evidence | |
| evidence --> promotion[manual Promote Worker] | |
| promotion --> latest[Registry latest + GitHub Release] |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/sops/release.md` around lines 57 - 77, Update the release flow
description around the create-release stage and its “Public GitHub prerelease”
output to distinguish release types: describe staged Registry releases and
semantic prereleases as GitHub prereleases, while stable workers with
interface_smoke disabled are documented as normal GitHub Releases. Keep the
existing workflow stages and outputs unchanged apart from this release-type
clarification.
ytallo
added a commit
that referenced
this pull request
Aug 4, 2026
The provider-deepseek worker (#691) resolves its credential from the DEEPSEEK_API_KEY env on the router, but the E2E workflows only forwarded the anthropic/openai/zai secrets, so a deepseek subject or judge could never discover models in CI. Declare the optional secret in the reusable workflow, forward it from every caller (including the release pipeline's E2E gate from #692), add the provider to the Rust cache workspaces, and export it in both run steps.
ytallo
added a commit
that referenced
this pull request
Aug 4, 2026
) The provider-deepseek worker (#691) resolves its credential from the DEEPSEEK_API_KEY env on the router, but the E2E workflows only forwarded the anthropic/openai/zai secrets, so a deepseek subject or judge could never discover models in CI. Declare the optional secret in the reusable workflow, forward it from every caller (including the release pipeline's E2E gate from #692), add the provider to the Rust cache workspaces, and export it in both run steps.
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@nextand keep the GitHub Release marked as a prerelease until promotion@latestWhy
Publishing directly to
@latestmakes a new worker version the default before its published artifacts have been exercised. This adds an explicit candidate stage, preserves immutable versioned artifacts, and requires successful evidence before a manual promotion.Impact
Create Tag now publishes stable worker versions to
@next. Maintainers promote a validated candidate with the Promote Worker action using its worker, version, and Release run ID. Existing GitHub-only releases continue through the non-staged path.Validation
python -m pytest -q .github/scripts/tests— 176 passedactionlintshellcheck harness/tests/quickstart/run-ci.sh harness/tests/e2e/run-deployed-ci.shgit diff --checkRefs MOT-4299
Summary by CodeRabbit