Pipeline integrity: epic integration review, git isolation, spec-aware context & escalation resolve - #3
Merged
Merged
Conversation
Child tasks carry a `Touches:` file list and an `Out of scope`/`Notes` boundary (e.g. "Any read/write code — T-004"), but the engineer agent ignored it and edited files another task owned — scope creep the reviewer then rejected every attempt, escalating NOT_CONVERGING (observed on a DDL-only migration task that also rewrote question-logging app code). The prompt said "modify only the files required" but never tied that to the ticket's own boundary. Make it explicit: change only files in `Touches:`, never touch a file reserved for another task, and if the task can't be completed in isolation, state the needed out-of-scope change in Deviations (routing to a human) rather than silently doing a sibling task's work. Adds a matching self-verification item. Reviewer side is unchanged — it already catches this via the intent-drift rubric; this stops the engineer from producing it in the first place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`git diff HEAD` does not show untracked files, so an agentic IMPLEMENTATION that
only ADDS files — a migration, a new module — produced an empty diff. Two
consequences, both observed while driving a real epic:
1. The "agentic implementation produced no repository changes" guarantee would
fail a run that did legitimate work.
2. New files stayed untracked, so CODE_REVIEW and QA saw them as "untracked in
git" and could not tell whether they were part of the change set — one QA
preflight blocked the epic on exactly that question.
Add GitGateway.stageAll (`git add -A`) and call it before diffing in the agentic
path. Staging both makes new files visible to `git diff HEAD` and leaves them
tracked for downstream review. Nothing is committed — the change set stays one
reviewable diff.
Engineer prompt updated to match: create files normally (AEOS stages them), do
not `git commit`, and list every new file in the Code Changes table.
654 tests, typecheck, lint, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the staging fix, which regressed reviews: `git add -A` stages the
whole working tree, and because AEOS never committed the source repo, that tree
accumulated every prior task's work plus unrelated local files. A docs-only task
was then reviewed against the entire epic and rejected for an "orphan
unacknowledged file" (a user's .claude/agents/*.md from days earlier).
Root cause: `git diff HEAD` on a never-committed tree is not "this ticket's
diff", it is "everything since the epic began".
Two commit points, both via the new GitGateway.commitAll (stages, commits only
if dirty, never empty):
- Baseline, before an agentic worker's first attempt: whatever is already
uncommitted becomes its own commit, so the worker's changes are the only
thing in `git diff HEAD`. Retries skip it, or a rejected attempt's work
would be baselined away mid-loop.
- Task completion, when a TASK advances to DONE: its work becomes one
properly-named commit, leaving the next task a clean tree.
Nothing commits mid-pipeline: CODE_REVIEW reads `git diff HEAD`, so an earlier
commit would leave the reviewer with "No changes detected". Epics are skipped —
they produce artifacts, not code.
commitAll verified against real git (no empty commits, untracked files tracked,
clean tree is a no-op). 658 tests, typecheck, lint, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Postmortem of the first full-epic AEOS run, which shipped two spec-fidelity defects past every green per-task gate: a Langfuse idempotency bug (T-001's verified work was silently lost by the shared uncommitted working tree) and a D-6 delivery-path violation baked in at decomposition. Roots the failures in seven causes (RC-1…RC-7). The tech spec designs the fixes: durable per-task git isolation, an epic-level INTEGRATION_REVIEW stage over the assembled diff, injecting the epic PRD/tech spec into child-task context, spec-traceability in the breakdown rubric, a non-mocked integration-test requirement, checkable cross-task dependencies, and resolving every escalation through an editable escalation.md (like preflight questions). No code yet — design only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…(WI-4, WI-5) Pure rubric/data changes from the pipeline-integrity spec: - WI-4: new spec-traceability rubric for TASK_BREAKDOWN — every tech-spec decision maps to a task, no task silently contradicts a decision (D-6/D-9.1), and deferred cross-task behavior is actually owned by a sibling. Wired into task-breakdown.yaml's reviewerRubrics. Its validation example reproduces the exact STAN-1 drift and shows the rubric FAILs it. - WI-5: new "Integration Seam Exercised" criterion in the code-structure rubric — flags tests that mock the very collaborator a change depends on and assert against the mock (the "false confidence" that hid the dropped Langfuse id). No code paths change; template-source test green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ntext (WI-3) A child task never saw the epic's PRD or tech spec — ContextAssembler read only the task's own artifact dir — so it could not implement against, or avoid contradicting, the decisions that scoped it (D-6/D-9.1 drift in STAN-1). The assembler now parses the parent id from the ticket document's AEOS metadata (the "- Parent:" line, already written for every task) and, for a task, loads the parent epic's *-prd.md and *-tech-spec.md into a new AssembledContext.epicContext. The prompt builder renders it first, framed as "Epic Specification (authoritative — do not contradict its decisions)". No new port dependency — the parent id rides the ticket content already in hand. Engineer self-verification gains a "consistent with epic decisions" item so the injected spec is actually checked against. 662 tests, typecheck, lint, build green; template-source green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ation.md (WI-7)
Generalizes the preflight questions round-trip to every escalation. A run that
ends ESCALATED now writes <ticketId>-escalation.md (reason, message, See:
pointer, and a fenced ## Response block). The operator writes their decision and
runs `aeos ticket resolve <id>`, which:
- parses the response (ignoring guidance comments),
- writes it as <ticketId>-resolution.md — picked up as authoritative prior
context on the retry (so the worker honors "use response_url per D-6",
"T-001's id-forwarding was lost, re-add it", etc.),
- returns the ticket to READY (which also clears the stored escalation),
- commits the resolution + escalation artifacts.
Mirrors `ticket answer`: the same not-modified confirmation guard and the same
compensate-on-commit-failure. Preflight BLOCKED still uses questions.md +
`ticket answer`; only true ESCALATED gets escalation.md. `ticket show` now points
the operator at whichever flow applies.
New: escalation-document service, TicketResolve use case/port/command, container
+ CLI wiring. 671 tests, typecheck, lint, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The orchestrator now gives each epic its own feature branch before any task runs: ensureOnBranch(`aeos/<epicId>`) plus a base tag (`aeos-base/<epicId>`) at the branch point. Task commits accumulate on that branch in isolation, and the tag lets the whole-feature diff (base..HEAD) be computed later without any stored state — which WI-2's integration review consumes. - GitGateway gains isRepo, ensureOnBranch, tagHere, diffRange, refExists; verified against a real repo (branch created, base tagged, range diff shows a committed file, tag idempotent). - OrchestratorUseCase takes GitGateway and sets isolation up once the run is cleared to proceed. No-op when the project path is not a git repo, or when AEOS_GIT_ISOLATION=none. - ensureOnBranch preserves uncommitted work (create-from-HEAD or checkout); nothing is destructive. 673 tests, typecheck, lint, build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mbled feature (WI-2) Adds the one stage that reviews the WHOLE feature against the epic's PRD and tech spec — the check per-task review structurally cannot do, and the gap that let the STAN-1 defects ship. Epic pipeline is now: … → TASK_BREAKDOWN → INTEGRATION_REVIEW → DOD_GATE → DONE Once every child task is DONE, the epic enters INTEGRATION_REVIEW (the existing children-DONE join now gates entry here via nextColumnFor). Its worker, a new integration-reviewer agent, reviews the assembled feature — the base..HEAD diff of the epic branch (WI-1's tag), injected by ContextAssembler — against the epic PRD/tech-spec, tracing every decision (D-N) and acceptance criterion into the diff, checking seams for real (non-mocked) tests, and verifying deferred hand-offs are present. The standard reviewer grades that report against a new spec-fidelity rubric. advanceMode is manual: a human reads the integration verdict before the DoD gate regardless of the pass. - column.ts: INTEGRATION_REVIEW added to the enum, COLUMN_ORDER, EPIC_COLUMN_ORDER. - ContextAssembler: INTEGRATION_REVIEW injects diffRange(aeos-base/<epic>, HEAD), falling back to the working-tree diff if the tag is absent. - Templates: integration-review.yaml column spec, integration-reviewer-agent.yaml, rubrics/drift/spec-fidelity.md (its validation example rejects a review that would have passed the STAN-1 defects). - Column-spec loader map, CLAUDE.md pipeline, and column tests updated. 685 tests, typecheck, lint, build, template-source green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…N_REVIEW - Tech spec status table: all seven work items done, with the two design deltas (env opt-out vs project.json; base git tag vs stored base_ref). - README: epic pipeline diagram + tables include INTEGRATION_REVIEW, the epic advance example runs it, and `aeos ticket resolve` is documented. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the gaps found in the STAN-1 postmortem — a full-epic run that shipped two
spec-fidelity defects past every green per-task gate. Design in
docs/aeos-pipeline-integrity-tech-spec.md; analysis indocs/aeos-postmortem-stan1.md.What broke, and what now catches it
idwork was silently lost (no per-task commit for most ofthe epic → fragile shared working tree). → WI-1 per-epic branch isolation +
WI-2 integration review tracing decisions into the real diff.
chat.updatevs the spec'sresponse_url(D-6) drifted at decomposition andno stage rechecked it against the tech spec. → WI-4 breakdown spec-traceability
rubric + WI-3 epic spec injected into child context + WI-2 whole-feature review.
WI-5 integration-seam rubric + WI-2 seam/test-reality check.
Changes
aeos/<epicId>branch +aeos-base/<epicId>tag; opt out with
AEOS_GIT_ISOLATION=none.GitGatewaygains isRepo/ensureOnBranch/tagHere/diffRange/refExists.INTEGRATION_REVIEWcolumn over thebase..HEADfeature diff vsPRD/tech-spec, with an integration-reviewer agent and spec-fidelity rubric; manual advance.
ContextAssemblerinjects the parent epic's PRD/tech-spec into child-task context.escalation.md+aeos ticket resolve: respond in a file and resume, response injected as context.New epic pipeline
… → TASK_BREAKDOWN → INTEGRATION_REVIEW → DOD_GATE → DONENotes for reviewers
aeos project syncto pick up the new column spec, agent, and rubrics.🤖 Generated with Claude Code