fix(pr-swarm): react to first failing CI check instead of full matrix - #85
Merged
Merged
Conversation
Lanes previously waited a flat ~10-minute quiet window before re-checking anything, and could read an already-red PR or an admin-gated `gh run rerun` as a reason to stall. Neither is necessary: pushing a fix always triggers a fresh CI run on its own. - Lane orientation now runs `gh pr checks` alongside `gh pr view` and treats a PR that already has failing checks or open feedback at launch as normal starting state to fix immediately, not a reason to wait. - The watch loop polls per-job status instead of only the run-level aggregate, and reacts to the first individually-failing check or new feedback rather than waiting for every check in a run to finish. - verification-gates.md documents the per-job polling rationale and clarifies `gh run rerun` (which requires admin rights) is not part of the normal fix-and-push cycle.
Ties the reference-doc behavior change into the top-level skill: Phase 6 now states that an inherited red PR at launch is normal input, not a swarm-level stop condition, and points at the independence check as the actual gate. Adds matching Error Handling and Red Flags entries, and updates the verification-gates.md reference-table summary. Bumps metadata version 1.0.0 -> 1.1.0 for the behavioral change.
Adds an eval case asserting the skill still activates when a targeted PR already has failing checks and open feedback, and that the response treats it as immediate work rather than a blocker.
Output of scripts/generate_manifest.py after the pr-swarm SKILL.md version bump.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pr-swarmlanes previously treated CI/feedback state too passively:state/mergeable/mergeStateStatus— it never looked at whether the PR already had failing checks or open review feedback, and nothing told it to act on that immediately.gh run rerunrequiring repo admin rights — a lane could plausibly read "can't force a fresh CI run" as a reason to stall, when the normal fix-and-push cycle never depends on that command at all.This PR makes lanes react to what's already visible instead of waiting on a clean baseline or a full CI matrix, and documents why an admin-gated rerun is a non-issue for the normal flow.
What changed, by commit
fix(pr-swarm): react to first failing CI check instead of full matrix—references/lane-prompt-template.md,references/verification-gates.mdORIENTATIONnow runsgh pr checks {PR_NUMBER}alongsidegh pr view, and instructs the lane to treat a PR that already has failing checks or unresolved feedback at launch as normal starting state — fix it as the first action, don't wait for a clean baseline.WATCH LOOPrewritten from waiting the full ~10-minute quiet window before checking anything, to polling per-job status (gh pr checks) and reacting to the first individually-failing check or new feedback, without waiting for the rest of that run's still-pending checks. The passive quiet-window wait now only applies once nothing is currently actionable.verification-gates.mdsection "React to the first failing check, not the full matrix": explainsgh pr checksis per-job, not run-level, and that a lane rarely needs to force a fresh CI run at all since pushing a fix triggers one automatically —gh run rerun(admin-gated) is reserved for the narrow flaky-retest case, not the default response to red CI.docs(pr-swarm): note first-failure reaction in skill and bump version—SKILL.mdverification-gates.mdreference-table description to mention the new first-failure-reaction content.metadata.version1.0.0 → 1.1.0 for the behavioral change.test(pr-swarm): cover pre-existing CI failure at swarm launch—evals/cases.yamlpr_with_existing_ci_failures: a swarm request where one PR already has failing checks and open feedback at launch. Asserts the skill still activates and the response treats the existing state as immediate work, not a blocker.chore(pr-swarm): regenerate manifest for version bump—manifest.yamlscripts/generate_manifest.pyoutput reflecting thepr-swarmversion bump. No other manifest content changed.Verification
uv run python scripts/generate_manifest.py— regenerated cleanly, diff limited to the version bump line.uv run python scripts/validate_evals.py— all packages pass, including the new eval case (73 skills validated).uv run python scripts/evaluate_package.py --path skills/pr-swarm— 96/100 (PASS), unchanged from before this change.python -c "import yaml; yaml.safe_load(open('skills/pr-swarm/evals/cases.yaml'))"— new eval case parses correctly (6 cases total).Skill Evaluator Results
Checklist
SKILL.mdhas valid YAML frontmatter withnameanddescriptionscripts/validate_evals.py,scripts/evaluate_package.py)SKILL.mdresolve to existing files