diff --git a/CHANGELOG.md b/CHANGELOG.md index aa2f66f0..077aeca3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,13 @@ breaking changes may land in a minor release. ### Added +- **A failed re-arm commits probe now journals `rearm-commits-probe-failed`** (DW-81). + The warn-only probe that lists the commits an abandoned attempt left below the re-drive's + new baseline used to swallow its `GitError` and write nothing — byte-identical to finding + no commits at all. It now records the baseline and the typed error, and both operator + surfaces render it through `runs.rearm_event_notice`. Advisory: it does not hold the + resume. + - **Review-gate verify commands are journalled** (#656, partial). The three review gates (`verify_review`, `verify_review_stories`, `verify_review_bundle`) now emit one `verify-command-result` per command, `verification_stage: "review"`, sharing the story's @@ -22,6 +29,18 @@ breaking changes may land in a minor release. line. Deliberately not widened to `run_verify_commands`, which has three legitimate callers on two roots. +- **Source-scan parity guards for three invariants previously held only by docstring prose** + (DW-65, DW-66, DW-82). The task-directory artifact names move to one shared + `journal.TASK_CYCLE_ARTIFACTS` that both adapters and `resolve._gather_escalations` iterate, + and `tests/test_portability_guard.py` gains three detectors: a bare artifact literal outside + that constant, a session task id composed outside `engine._session_task_id`, and a journal + field name that neither `diagnostics`' redaction tables route nor the benign inventory + declares. Each carries positive and negative probes so a detector that stops detecting cannot + read as green, and an unresolvable `journal.append(**splat)`, a journal write whose kind is + not a string literal, and a benign entry whose producer has been deleted all fail loud rather + than being skipped. Journal field routing is graded per kind where `diagnostics` routes per + kind, and a declared forwarder's call sites (`plugins/bus.py::_log`) enter the inventory. + - **`repo_root` in run `state.json`** (#716). A run records the git root its code work happens in, so an out-of-process reader — `bmad-loop resolve`'s re-arm — uses the tree the run measured instead of re-deriving one. A `state.json` written before the field existed degrades to the @@ -58,6 +77,12 @@ breaking changes may land in a minor release. ### Changed +- **`bmad-loop diagnose --json` reports `schema_version: 3`.** Replacing a journal-entry value + with a presence key is a payload break under the additive-only rule, and the redaction fixes + above make two: a consumer reading `entry["question"]` on `decision-pending`, or an + off-schema key on `preference-escalation`, finds a `_present` boolean instead. Exactly + what minted v2 for `patch` / `stashed_to`. Structure is otherwise unchanged. + - **A story's `verification_sequence` now numbers its review passes too**, so the ordinals a `post_dev_verify` handler receives shift: for an unchanged run whose review gate sits between the dev and repair legs, the `fix` pass moves from 2 to 3. The ordinal was always documented @@ -207,9 +232,82 @@ breaking changes may land in a minor release. ### Fixed +- **An artifact the escalation walk cannot stat is recorded as unreadable rather than absent** + (DW-11). `resolve._gather_escalations` classified each task-cycle artifact with + `Path.is_file()` outside its guard, and that probe's answer to EACCES splits by interpreter. + Through 3.13 it re-raises anything outside `pathlib._IGNORED_ERRNOS`, so an artifact under an + unreadable directory escaped `build_context` and `cmd_resolve` to the top-level backstop as + `error: [Errno 13] ...` — a read fault ending the interactive command this reader's contract + says it must survive. On 3.14, where the body became `os.path.isfile`, the same fault was + swallowed as absence instead: the skip sink stayed empty, so the caller recorded coverage and + `escalations_resolved_upto` withheld every CRITICAL entry under that directory permanently. + Classification is now `stat`, which answers with an errno — ENOENT and ENOTDIR are genuine + absence and still cost nothing, while EACCES, EIO, ESTALE and EBADF join the shown-side skip + sink. ELOOP moves with them — the one reading that changes on every interpreter rather than + one, since a symlink cycle answered False through 3.13 (ELOOP is in the ignored tuple) and on + 3.14 (`os.path.isfile` swallows it): a degrade withholds coverage rather than being laundered + into a durable claim. The regular-file check stays, because `stat` succeeds where `is_file()` + answered False — without it a directory at an artifact path would raise `IsADirectoryError`, + and a FIFO would block the read forever, wedging the interactive command. +- **An aborted re-arm no longer leaves the spec re-armed against an escalated task** + (DW-79, DW-83, DW-85). `runs.rearm_escalation` published the status flip and stripped the + stale `## Auto Run Result` about 250 lines before `save_state`, and only two of the aborts + in that window undid those writes — a failing `journal.append` from the stale-restore + residue pass, a non-git fault from the commits probe, or a failing `save_state` each escaped + with the spec flipped on disk while persisted state still said ESCALATED. The window is now + one transaction with `save_state` as its single commit point: any fault — an interrupt + included, which is why the guard catches `BaseException` — restores the spec's BYTES to what + the re-arm found and re-raises the original fault unchanged. Clearing a stories sentinel is + deliberately outside that scope: it unlinks a file rather than writing spec bytes, and + `_clear_sentinel` already preserves a copy and is idempotent on retry. The rollback journals + `rearm-aborted` carrying `restored`, `unchanged` (proved byte-identical), `failed` or + `unknown`, which `resolve` and the TUI render through the one shared routing table — so the + residue notices they echo from a `finally` can no longer describe files as excluded from a + baseline that was never saved, and an outcome the undo could not confirm (the cleared + sentinel among them) reports the abort without claiming the file is intact. A rollback that + cannot write still raises, naming the possibly part-written spec to restore from git _or_ + your own copy — an untracked or out-of-checkout spec has no committed version to recover — + with the original fault kept in the exception chain. The undo picks its writer the same + lexical way the three spec writers it undoes do, so a spec in an artifacts folder configured + outside the checkout is restored rather than refused; it declines to write at all when the + spec's bytes could not be CAPTURED from a file that is there and that the re-drive will + actually read, since a transient read fault followed by successful writes would leave a + published flip with nothing to put back; and it leaves a re-arm whose `save_state` + demonstrably committed alone, because that rename can be interrupted on the way out and + undoing the spec beneath it mirrors the same defect. An ORDINARY failure of the abort + record's OWN journal write is suppressed whatever its type, so an observation that cannot + be made never replaces the fault the operator is being told about — an interrupt still + leaves, since by then the rollback has already run and the operator asked to stop. + +- Stop an LLM-authored preference escalation from aborting the review leg. `_review_and_commit` + splats a review session's own `result.json` escalation entries into `journal.append`, so a + result.json carrying a `kind` or `story_key` key raised `TypeError: got multiple values for +argument` and failed the story; a `ts` key did not raise and instead silently replaced the + entry's real timestamp, skewing every relative offset a diagnostic dump derives from it. Those + three journal-owned names are now dropped before the splat. + +- Close three journal-field leaks into `diagnose`, all found by the new field-routing guard and + each reproduced through `diagnostics._scrub_entry`. On `preference-escalation`, whose keys are + LLM-authored (`engine._review_and_commit` splats a session's own `result.json` entries into + the journal), every key outside the record's declared `{type, severity, detail}` schema now + renders as `_present`; the key NAME still ships, a bound stated on the routing entry and + accepted rather than collapsed. `decision-pending`'s `question` joins the free-text drop set — + a multi-word question collapsed only by accident of the fallback forbidding spaces, while a + one-token one shipped verbatim; no operator surface loses it, since the TUI reads the raw + journal. And `story_keys` is aliased element-wise on `sweep-inflight-stranded`, which carried + raw bundle story keys because the value fell through to `scrub_json` — the identity on a list + of identifier-shaped strings — while the singular `story_key` beside it was already aliased; + a non-list value on any key-list field now fails closed instead of taking that same path. +- Stop a second resolve cycle re-presenting escalations the human already answered + (DW-11). Only a re-arm that accepted a `resolution.json` watermarks the session + trail; later cycles show what came after it and print how many were withheld. A + cycle whose walk could not read a session artifact records no coverage at all, so + a transient read fault no longer buries the escalations it hid. - Emit `diagnose --json` v2, replacing journal `patch` / `stashed_to` paths with `patch_present` / `stashed_to_present`, and silently degrade Git stale-commit probe failures while propagating non-Git faults. +- De-duplicate interactive-resolution escalations across repeated task IDs and mirrored + artifacts, and skip malformed artifacts without aborting resolution. - Prevent escalated sweep restarts from reusing abandoned session ids, and clear stale `escalation.json` artifacts when either adapter reuses a task directory. - Route interactive resolve task ids through the shared whole-composition sanitizer while diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 50bc2cb3..0ceb1e84 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -68,14 +68,14 @@ See [README.md](../README.md) for the narrative overview and [setup-guide.md](se - Completing a park: `bmad-loop confirm ` walks the outstanding actions one at a time, writes the spec's `## Operator Confirmation` audit section, advances spec and board to `done`, and commits the pair together with the park record's deletion. Nothing is re-driven — the agent-doable work was committed at park time; `--reverify` re-runs your `[verify]` commands first and a failure blocks the confirmation. Each park is a **committed per-story file** under `.bmad-loop/operator/`, written inside the story's commit window so it rides the park's own commit through the merge-back to every clone — a teammate, a fresh clone or CI can confirm a story parked elsewhere (#356). `validate` warns on drift in every direction (`operator.registry-stale`, `operator.actions-malformed`, `operator.park-record-missing`), and `confirm` refuses a drifted record. A park written before #356 lives in the machine-local `.bmad-loop/operator-actions.json`, which `confirm` still reads and prunes but nothing writes anymore — so an in-flight park from an older version stays confirmable on the machine that wrote it. - A confirmation is resumable. Every write is checked — the spec is read back from disk, so a story is never declared done over a write that did not land — and the park record is dropped last, so a failure part-way leaves the story findable. Interrupted between the spec writes and the board write, what survives is a signed-off spec at `done` with the entry still pointing at it; re-running `confirm` **finishes** that rather than refusing it as stale, with no second prompt and no second audit section (the section on disk _is_ the acknowledgment, and the check is fence-aware). It resumes equally from a board a human fixed by hand, which is what the failure message asks for — advancing an already-`done` board is idempotent. `--list`, `--json` (`resumable`, `confirmation_recorded`) and `validate` (`operator.confirm-interrupted`) name that state rather than calling it stale. - Dispatched sessions are told the sprint board is orchestrator-owned (#437) — the sibling of the park contract above, injected into the prompt the same way. The board advances as soon as dev verifies, but the story's single commit lands only after the review loop, so a session dispatched in between opens on an uncommitted, unattributed change to `sprint-status.yaml` with nothing in the repo naming its author (one read it as a spec violation, reverted it, and tripped the sign-off-regression gate on a story both sessions agreed was finished). Story dev prompts and the review prompts of sprint and sweep runs carry the same prohibition: never write the board, never revert it, and a row at `done` or `awaiting-operator` is the orchestrator's own bookkeeping — not a defect to fix, and not proof that the work is verified, deliberately, since the row is written _before_ the deterministic dev verification runs and a repair session opens on a red tree under a `done` row. Only the **review** prompt adds where to go instead: a story that cannot be finished without a human decision is finalized to `status: blocked` with a reason — the one hand-back that both withholds the commit and reaches a human, where any other non-terminal status just burns the review budget onto a defer that rolls the work back. A dev prompt gets no such invitation, because `blocked` halts the whole run — the exact failure park exists to avoid — and a dev session that cannot finish already has park. A deferred-work bundle's dev prompt carries nothing (a bundle has no board row) while a bundle's _review_ prompt does, since a sweep runs inside a project whose board exists and is just as revertible; every injected plugin-workflow session carries the prohibition too — `post_dev_phase`, `post_review_result` and `pre_commit_gate` all fire inside that same window — as its own `## Sprint board` section appended _after_ the session-gate hooks, so a plugin prompt rewrite cannot strip it, and without the `blocked` redirect for the same reason a dev prompt has none; stories mode carries none of it, having no board at all. -- Typed escalations: `CRITICAL` pauses the run + notifies (desktop + `ATTENTION` file); `PREFERENCE` is journaled and continues. +- Typed escalations: `CRITICAL` pauses the run + notifies (desktop + `ATTENTION` file); `PREFERENCE` is journaled and continues. A story's escalation trail is append-only and deliberately survives a re-arm (it is the run-dir audit a later resolve cycle reads), so a second `bmad-loop resolve` used to re-present every CRITICAL the story ever raised, interleaved with the new ones and with nothing marking which was which — against a resolve skill whose contract is singular. An interactive resolve session that records a `resolution.json` now **watermarks** the trail at its current length, and every later cycle hands the agent only the escalations recorded since; how many earlier ones were withheld is printed to your terminal, never added to the agent's `context.json` (the agent-facing contract is unchanged). The watermark moves only on a gesture that actually accepted a resolution — a resolve session that exited without writing one, `resolve --no-interactive`, and the TUI's Re-arm button all leave it where it stands. Leaving a watermark is not clearing it: a watermark already standing still filters on those paths, which show everything recorded since the last accepted resolution rather than the whole trail. That is where the bias is deliberate, and it is a claim about which GESTURES move the watermark: one that accepted nothing never moves it. Within a cycle that DID accept a resolution the watermark covers everything that cycle PRESENTED — it is stamped at the trail's length, not at the entries individually answered — so answering one of five escalations shown together retires all five. A task's watermark is reported as the `esc-upto` column of `bmad-loop diagnose`'s markdown task table, and as `escalations_resolved_upto` under `--json` (that is the key to grep in a support bundle), which is what explains a short `context.json` on a bug report. - A rejected dev attempt notifies too, with its reason (#640). RETRY was the only dev outcome that rejected an attempt silently, and it is the one that discards a completed implementation — the non-fixable leg resets the tree to baseline. The notice fires once per rejected attempt in an uninterrupted run (so ordinarily at most `max_dev_attempts` per story) and has no suppression knob of its own; it follows `[notify]` like every other notice. One attempt can raise it twice: the notice precedes the rollback, so a host that dies in between replays that verdict on resume and announces it again — treat the count as a floor on attempts rejected, not an exact tally. The reason is reduced to its first line and capped, with a `[…]` marker when it was trimmed, because a `Decision.reason` routinely carries a verify-output tail that would otherwise spill into `ATTENTION` and a desktop bubble; the untruncated reason stays in the `dev-decision` journal entry. It fires above the fixable/non-fixable split, so on a leg that goes on to pause for manual recovery the operator sees both notices. - Environment faults pause without burning budget (#194): a session whose coding CLI never reached the API — a verify command whose _environment_ is broken (`sh` reports rc `126`/`127`; on Windows a missing tool is caught by its `is not recognized` message or by resolving the command's leading token, and a command naming a file `cmd` cannot execute — a `.sh`, or any extension outside `PATHEXT`, which cmd hands to the file association and which exits `0` without running anything — is a fault rather than a silent rc `0` pass, #302; and on either OS a verify command whose child could not be started at all — most often because the directory it was to run in is missing, is a file, or cannot be searched, but any spawn-time `OSError` counts — is translated into the same fault instead of crashing the run, since no exit code exists to classify) **or** a session whose log matches the profile's `env_fault_patterns` (an `API Error … Connection refused`-class transport failure, or a provider quota/usage-limit refusal, that idled out the session clock) — pauses the run with the matched evidence instead of charging the attempt and deferring the story as if its code were broken. Re-arm restores the budget. Patterns are per-profile: `claude` seeds three, reproducing only complete error sentences its CLI was captured printing (connection loss, and the two captured provider 5xx refusals — statuses enumerated, never ranged, so an uncaptured `503` stays prose), so a story that merely writes _about_ a provider error cannot trip them (#507); `opencode` seeds a provider quota/rate-limit and connection pair (#323), matched against the `opencode serve` process's own stdout, which the model cannot write to; the other four profiles ship none. Each adapter matches them against the log named by its `ENV_FAULT_LOG_SUFFIX` — the tmux pane capture `logs/.log`, or `.server.out` (the `opencode serve` process's own stdout) for `opencode-http`, never that adapter's model-written transcript. A pattern is only sound against a log the model cannot write to; where that does not hold — the pane capture — the pattern has to reproduce a whole captured sentence, because an error token plus a cause on the same line is precisely the shape a story writing about the error emits, and that framing is what the guard now refuses (#507). A usage-limit / quota cause stays unseeded on the pane-capture profiles for the same evidentiary reason: no captured line exists for them (#323). Extend or disable them in a project profile overlay. - A session the multiplexer lost says so (#489). Sessions complete on a hook `Stop` or on window death, and a window is gone whether the CLI exited or something destroyed the whole mux session out from under the run — an external reaper, a concurrent prune or `bmad-loop stop`, an operator `kill-session`, a server crash, the host sleeping. Both are `crashed`, so the retry/defer reason an operator reads said only `dev session crashed` — pointing at the agent when the host was at fault. The crash verdict now asks whether the _session_ still exists and, when it does not, says so in the reason (`… session crashed: the multiplexer no longer reports the session, so the window's disappearance is not evidence the CLI exited`), as `session_vanished` on `dev-decision` and `fix-decision` either way, beside the routing each fed, on every role's `session-end` journal entry when it is true (the convention `env_fault` already uses there), and as a `session-vanished` breadcrumb in `session-lifecycle.jsonl`. The repair path carries it the same way: when fix attempts are exhausted the defer names the lost session instead of blaming the tree for repairs that never ran. The wording states what the evidence _withdraws_, not what it proves: `has_session` maps every nonzero backend result to False, so a negative lookup is "the backend did not confirm it" rather than proof the session is gone — enough to stop an operator reading window death as a CLI exit, not enough to name a destroyer. It composes with an environment-fault pause instead of being swallowed by it. A session reaped _after_ flushing its result still scores `completed` and is not diagnosed — it produced something. Diagnosis only — the routing is unchanged, and a retry re-creates the session. - CRITICAL resolution: `bmad-loop resolve ` opens an interactive resolve agent seeded with the escalation + frozen spec; you disambiguate, it re-arms the story (`escalated → pending`, spec reset to `ready-for-dev`) and resumes. `--no-interactive` skips to re-arm if you fixed the spec yourself. The re-arm advances the story's baseline in the **code tree** and is honest when it cannot: a failed advance is narrowed to typed git errors, journalled, echoed to stderr, and explicitly NOT followed by a re-stamp it did not earn, so - spec and task never silently agree on a stale sha (#640). A re-stamp that does overwrite a differing + spec and task never silently agree on a stale sha (#640). The re-arm's other warn-only git probe — the one that lists the commits an abandoned attempt left below the re-drive's new baseline — is honest the same way (DW-81): its Git failures journal `rearm-commits-probe-failed` and echo to the same surfaces, because that probe's silence is otherwise indistinguishable from a clean answer, and the absent warning is the operator's only sign that those commits are now a permanent starting point nothing will revisit. A re-stamp that does overwrite a differing claim records what it replaced, and warns on either leg: the record fires only when the spec claimed a baseline the run never recorded, which is the only remaining trace of a divergence the gate can no longer report. `spec_file` is persisted relative to a worktree for an isolated task, so every out-of-process reader @@ -126,6 +126,38 @@ Result` section. Every other spec keeps warn-and-continue, and the record says w the re-drive mounts from does not already hold this checkout's copy of those two files, so a correction already committed there resumes in one gesture, and an in-place re-drive never records at all — it reads the main checkout, which is where the resolve session runs. + The whole re-arm is one **transaction**, and what it covers is stated narrowly: the SPEC's + BYTES, from the first spec write to `save_state`. That save is the commit point — until it + returns the run still calls the story escalated, so any fault escaping the window in between + (a journal write that fails, a non-git fault from the stale-restore commits probe, an + interrupt during one of the three git probes, or the state write itself) used to leave a spec + flipped to the re-drive's status and stripped of its `## Auto Run Result` against a task + nothing had moved. Every one of them now restores the spec to the bytes the re-arm found and + re-raises the original fault unchanged, so the escalation stays armed. One in-window tree + change is deliberately outside that scope: clearing a **sentinel** unlinks the file rather + than writing spec bytes, and it is not re-created, because `_clear_sentinel` already + preserves a copy under `{run_dir}/sentinels/` and a retried resolve re-clears it + idempotently. The re-arm says which of those it did. `rearm-aborted` is journalled from the + rollback and echoed by both surfaces, carrying `restored` (a write had landed and was put + back), `unchanged` (the file was read and PROVED byte-identical — a refusal sequenced ahead + of every write), `failed` (the restore itself could not write, so the spec may be part-written + — that one raises rather than degrading, keeps the original fault in the exception chain, and + names the file to restore — from git _or_ your own copy, since the bytes it failed to write + are gone with the process and a spec is not necessarily tracked — in both the message and its + next step), or `unknown`. The last covers everything the undo could not confirm: the cleared + sentinel, a re-arm that resolved no spec path at all (the record then carries an empty + locator and the notice says `(none)`), and a spec that is gone or unreadable by the time the + undo looks, which it declines to re-create rather than fight whatever removed it. The + surfaces then report that nothing was persisted and the story is still escalated WITHOUT + claiming the file on disk is intact — an unconfirmed outcome must never render as the + reassuring one. Without this record the surfaces described the residue of a re-arm that had + been rolled back — files "excluded from the re-drive baseline" for a baseline never saved. + Two boundaries keep the undo honest. It refuses to write at all when it could not first + CAPTURE the spec's bytes from a file that is there — a transient read fault followed by + writes that succeed would otherwise leave a published flip with nothing to put back — and it + does NOT undo a re-arm whose `save_state` demonstrably committed, since that state write is + a single atomic rename whose call can still be interrupted on its way out, and rolling the + spec back underneath it would build the same defect mirrored. All of these warnings reach the TUI's re-arm as well as `resolve`'s — both route every kind through one shared table, so neither surface can silently learn a kind the other drops, though each still owns where it calls the echo from and the TUI drops the trailing "before diff --git a/src/bmad_loop/adapters/generic.py b/src/bmad_loop/adapters/generic.py index 643445b2..4071976f 100644 --- a/src/bmad_loop/adapters/generic.py +++ b/src/bmad_loop/adapters/generic.py @@ -32,7 +32,7 @@ from .. import devcontract, gates, runs from ..bmadconfig import ProjectPaths -from ..journal import LOGS_DIR +from ..journal import LOGS_DIR, TASK_CYCLE_ARTIFACTS from ..model import TokenUsage from ..policy import Policy from ..process_host import ProcessHostError, get_process_host @@ -543,10 +543,11 @@ def start_session(self, spec: SessionSpec) -> SessionHandle: (task_dir / "prompt.txt").write_text(spec.prompt + "\n", encoding="utf-8") # Task ids are supplied by the caller, so defensively reset cycle-scoped # outputs if one is reused. A silent session must not inherit a stale result. - (task_dir / "result.json").unlink(missing_ok=True) - # The sweep skill also writes escalation.json here, and - # `resolve._gather_escalations` reads it alongside result.json. - (task_dir / "escalation.json").unlink(missing_ok=True) + # The list is `journal.TASK_CYCLE_ARTIFACTS` rather than two literals here: + # `resolve._gather_escalations` reads the same names back, so a third + # artifact must not be able to reach the reader while missing this adapter. + for artifact in TASK_CYCLE_ARTIFACTS: + (task_dir / artifact).unlink(missing_ok=True) self._ensure_session(spec.cwd) # Stamped before launch: hook events carry wall-clock ns, and diff --git a/src/bmad_loop/adapters/opencode_http.py b/src/bmad_loop/adapters/opencode_http.py index df13fe3b..032c4f81 100644 --- a/src/bmad_loop/adapters/opencode_http.py +++ b/src/bmad_loop/adapters/opencode_http.py @@ -146,7 +146,7 @@ from .. import gates from ..bmadconfig import ProjectPaths -from ..journal import LOGS_DIR +from ..journal import LOGS_DIR, TASK_CYCLE_ARTIFACTS from ..model import TokenUsage from ..policy import Policy from ..process_host import ProcessHostError, get_process_host @@ -624,10 +624,11 @@ def start_session(self, spec: SessionSpec) -> SessionHandle: (task_dir / "prompt.txt").write_text(spec.prompt + "\n", encoding="utf-8") # Task ids are supplied by the caller, so defensively reset cycle-scoped # outputs if one is reused. A silent session must not inherit a stale result. - (task_dir / "result.json").unlink(missing_ok=True) - # The sweep skill also writes escalation.json here, and - # `resolve._gather_escalations` reads it alongside result.json. - (task_dir / "escalation.json").unlink(missing_ok=True) + # Iterating `journal.TASK_CYCLE_ARTIFACTS` is what makes the parity with + # GenericAdapter.start_session structural instead of a claim in a test + # docstring: both adapters and `resolve._gather_escalations` share one list. + for artifact in TASK_CYCLE_ARTIFACTS: + (task_dir / artifact).unlink(missing_ok=True) # Same hazard, same reason, for the file the #194 tail scan reads (mirrors # GenericAdapter.start_session, which unlinks its pane tee here). This one # bites hardest on the path the classifier exists to serve: an env fault diff --git a/src/bmad_loop/cli.py b/src/bmad_loop/cli.py index ef81a5a1..2c9e0817 100644 --- a/src/bmad_loop/cli.py +++ b/src/bmad_loop/cli.py @@ -2965,10 +2965,16 @@ def _resolve_restore_patch( def _echo_rearm_events(run_dir: Path, before: list[dict[str, Any]] | None) -> bool: - """Surface the events a just-completed re-arm journaled: the `stale-restore-*` - residue of the restore attempt it abandoned (runs._stale_restore_residue), and the - `rearm-*` records the status flip, the advance and the re-stamp write. The commits - variant is the one the human must act on — nothing else will. + """Surface the events a just-completed re-arm journaled: the residue of the restore + attempt it abandoned — the `stale-restore-*` records AND `rearm-commits-probe-failed`, + all written by `runs._stale_restore_residue` — and the `rearm-*` records the status + flip, the advance and the re-stamp write. Split by PRODUCER, not on the prefix, + because the prefix does not partition them: the commits probe's failure record is + spelled `rearm-*` for the re-arm it degrades, not for the abandoned restore it + measures. The commits pair is what the human must act on — nothing else will tell + them — and it takes both, because `stale-restore-commits` is written only when the + probe ANSWERED: without its twin, that record's absence reads as "clean" whether or + not anyone could tell. Named for the re-arm, not for the stale restore: it began as a `stale-restore-*` echo and now carries the baseline family too, so a name from the narrower era @@ -3098,10 +3104,21 @@ def cmd_resolve(args: argparse.Namespace) -> int: print(err, file=sys.stderr) return 1 + # DW-11: whether THIS gesture accepted a resolution, which is what gates the + # `escalations_resolved_upto` watermark in `runs.rearm_escalation`. False here + # covers `--no-interactive` deliberately: that path accepted nothing IN THIS + # GESTURE (the human may have fixed the spec by hand, but nothing recorded which + # escalations that answered), so the next cycle shows everything — today's + # behavior, and the safe direction. Not derived from `resolution.json`: the marker + # survives the re-arm that consumed it, so its presence says nothing about this + # gesture. + resolution_recorded = False if args.interactive: adapters = _make_adapters(project, run_dir, pol) model = pol.adapter.resolved("dev").model - resolve.build_context(state, run_dir, story_key, isolation=pol.scm.isolation) + _ctx_path, withheld, unreadable = resolve.build_context( + state, run_dir, story_key, isolation=pol.scm.isolation + ) print(f"launching resolve agent for {story_key} — converse, fix the spec, then exit…") try: produced = resolve.run_session( @@ -3123,6 +3140,46 @@ def cmd_resolve(args: argparse.Namespace) -> int: file=sys.stderr, ) return 1 + # DW-11, second half. `produced` alone is not enough to record coverage, + # because the watermark `rearm_escalation` stamps is `len(task.sessions)` — it + # covers every recorded session, INCLUDING the ones whose artifacts this walk + # could not read. `_gather_escalations` degrades on those by design (an + # observation path must not raise out of an interactive command), but the + # watermark turns that transient silence into a durable claim: the next cycle + # reads the file fine and withholds it as already answered. So a skipped + # artifact withholds COVERAGE instead. The cost is one repeated presentation; + # the alternative cost is an escalation nobody ever sees. + resolution_recorded = bool(produced) and not unreadable + # DW-11. Reported to the operator, never into `context.json`: filtering the + # agent's list silently would trade one misleading surface for another — the + # human would have no way to tell "nothing else was ever raised" from "the rest + # is hidden". Worded for what the code can prove: these entries were PRESENTED + # to an earlier resolve cycle that recorded a resolution — not that any + # particular one of them was individually answered. + # + # Printed here rather than beside the context build, because until + # `run_session` returns without `NotImplementedError` this adapter is not known + # to support an interactive session at all — and an operator whose command is + # about to fail must not be told escalations were withheld from an agent that + # never launched. + if withheld: + print( + f"{withheld} earlier escalation(s) for {story_key} were not shown to the " + "agent: they were presented to an earlier resolve cycle that recorded a " + "resolution" + ) + if produced and unreadable: + # Only when a resolution WAS produced: with nothing recorded the watermark + # would not have advanced anyway, and reporting a withheld coverage the + # operator never had is noise. Counts, not paths — the operator's action is + # the same for one unreadable artifact as for five, and the run-dir names + # are not theirs to chase. + print( + f"{unreadable} session artifact(s) for {story_key} could not be read, so " + "this resolution was NOT recorded as covering the escalations they hold " + "— the next resolve will show every escalation for this story again", + file=sys.stderr, + ) if not produced: print( f"no resolution recorded for {story_key} (agent did not write resolution.json)", @@ -3227,6 +3284,7 @@ def cmd_resolve(args: argparse.Namespace) -> int: story_key, restore_patch=restore_patch, isolated_redrive=pol.scm.isolation == "worktree", + resolution_recorded=resolution_recorded, ) except runs.RearmError as e: print(f"error: {e}", file=sys.stderr) @@ -3235,8 +3293,9 @@ def cmd_resolve(args: argparse.Namespace) -> int: # In the `finally`, not after the `try`: `_stale_restore_residue` journals # BEFORE the re-stamp block that raises `RearmError`, so on that path the # records were already written and returning early threw them away — including - # `stale-restore-commits`, the one record whose whole point is that nothing - # else will tell the human. An abort is when that residue matters most: the + # the commits PAIR (`stale-restore-commits` when the probe answered, + # `rearm-commits-probe-failed` when it could not), whose whole point is that + # nothing else will tell the human. An abort is when that residue matters most: the # re-arm half-ran and the operator has to decide what to do with the tree. hold_resume = _echo_rearm_events(run_dir, before_entries) print( diff --git a/src/bmad_loop/diagnostics.py b/src/bmad_loop/diagnostics.py index cb7ac2fd..99c72ee6 100644 --- a/src/bmad_loop/diagnostics.py +++ b/src/bmad_loop/diagnostics.py @@ -51,7 +51,7 @@ from typing import Any from . import __version__, sanitize -from .journal import VERIFY_DIR, Journal, load_state +from .journal import SELF_MINTED_FIELDS, VERIFY_DIR, Journal, load_state from .model import RunState, StoryTask from .platform_util import walk_files_unlinked @@ -71,7 +71,22 @@ # the fence is gone and json.loads fails. Bump only on a payload break. # v2 replaces journal-entry `patch` / `stashed_to` values with the presence keys # `patch_present` / `stashed_to_present`. -SCHEMA_VERSION = 2 +# v3 does the same thing to two more journal-entry values, which is the same +# payload break for the same reason: `question` (on `decision-pending`) becomes +# `question_present`, and on `preference-escalation` EVERY key outside the +# `{type, severity, detail}` schema becomes `_present` — see +# `_JOURNAL_KIND_SCHEMAS`. A consumer reading `entry["question"]` finds it gone, +# so this is a break under `machine.py`'s additive-only rule, exactly as v2 was. +# The third change shipping with it — `_JOURNAL_KEYLIST_FIELDS` failing closed on a +# non-list value — is deliberately NOT part of this rationale, but the reason is the +# JSONL ROUND-TRIP, not the call sites. Two producers do pass a tuple +# (`sweep.py`'s `dw_ids=(decision.id,)` and `dw_ids=tuple(task.dw_ids)`), so a survey +# of producers would be the wrong argument and is false as such. What makes it a +# non-break is that `_scrub_entry` never sees a producer's object: entries are +# serialized to `journal.jsonl` and read back, and JSON has no tuple type, so every +# sequence arrives as a `list` and takes the same arm it always did. The new arm is +# reachable only by a shape no round-tripped entry can hold. +SCHEMA_VERSION = 3 DEFAULT_JOURNAL_CAP = 200 # Subdirectories whose mere existence/size is diagnostic but whose CONTENTS are @@ -118,6 +133,30 @@ # single record: alias one and leave the other and a dump pseudonymizes half a # comparison, which is worse than either doing both or doing neither. "overwritten": "commit", + # A THIRD spelling of a baseline sha, journalled by `runs._stale_restore_residue` + # on BOTH of its commit records: `stale-restore-commits` (the probe answered, and + # these shas sit above it) and `rearm-commits-probe-failed` (the probe could not + # answer). Routing is by NAME, so this one entry gives that one value ONE alias + # across both kinds — which is also why the producer was not respelled to the + # already-routed `baseline`: that would give one sha two spellings and two + # aliases in a single dump. + # + # Routed for CORRELATION, not to stop a leak — this table's header states that + # purpose ("pseudonymized, not dropped, so events stay correlatable") and it is + # the whole reason this entry exists. Left unrouted a real sha does NOT ship + # verbatim: `_scrub_str` applies `looks_like_secret` AFTER `looks_like_identifier`, + # and a real 40-hex sha clears the length+entropy bar, so the fallback renders + # `` — USUALLY, and the exception is the point. Real shas + # straddle that check's bar: measured over this repo's own history (1790 commits, + # 2026-08-31), about one sha in twenty-five is NOT caught and ships verbatim. So + # routing closes a real, intermittent leak. On the ~96% the fallback does catch it + # buys the other thing this table exists for: `` is safe and + # useless — an operator can no longer tell that the probe-failure record and the + # commits record name the SAME baseline, which is the one comparison those two + # kinds exist to support. That the name ALSO had to leave the routing guard's + # benign inventory follows from that inventory's own rule: a name carrying a sha + # belongs in a `diagnostics` table. + "old_baseline": "commit", # A spec name IS the customer's feature name — `Pseudonymizer`'s own docstring # has always listed "spec filenames" among what it exists to alias, so the # omission here was a routing gap, not a policy. A producer that journals a @@ -131,18 +170,25 @@ # whose epic could not be resolved. See `_JOURNAL_BASENAME_NAMESPACES` for why # the value is normalized first: the producers do NOT agree on a bare basename. "spec": "spec", - # The same value also arrives under a second field NAME. `runs.rearm_escalation` - # is the only journal producer of `spec_file`, across FOUR kinds — - # `rearm-spec-write-unreachable`, `rearm-spec-flip-skipped`, - # `rearm-baseline-restamp-skipped` and `rearm-baseline-restamped`. Routing is by - # field NAME, not by kind, so the list is documentation rather than a gate — but an - # enumeration that undercounts is how the next reader concludes a kind is unrouted. + # The same value also arrives under a second field NAME, from the re-arm family in + # `runs` — FIVE kinds, and no longer from a single function. `rearm_escalation` + # writes four of them (`rearm-spec-write-unreachable`, `rearm-spec-flip-skipped`, + # `rearm-baseline-restamp-skipped`, `rearm-baseline-restamped`); the fifth, + # `rearm-aborted`, is written by `runs._rollback_rearm` from the transaction guard's + # error path — a DIFFERENT function, which is why "the only producer" is no longer + # the right shape for this note. Routing is by field NAME, not by kind, so the list + # is documentation rather than a gate — but an enumeration that undercounts is how + # the next reader concludes a kind is unrouted, so it is corrected rather than + # left to age. `rearm-aborted` also carries `error` (dropped as free text) and + # `rollback`, a literal enum string declared benign in the routing guard. # `engine._park_awaiting_operator` passes # `spec_file=` to `operatoractions.record_park`, which is a record file, not the # journal. So the divergence is BETWEEN FIELDS, not between two producers of this # one — but BOTH fields are mixed-shape, and neither is the reliable one: # Both fields now journal an absolute path wherever they carry one: `spec_file` - # through `str(task_spec_path(...))` on all four kinds, and `spec` through + # through `str(task_spec_path(...))` on all five kinds — `rearm-aborted` forwards + # the same already-anchored value, and writes `""` rather than a story key when the + # aborted re-arm never resolved a spec path — and `spec` through # `engine._operator_spec_path` (which anchors `checkpoint-pause` the same # way) alongside engine's already-absolute reconcile and marker-repair kinds. Same # value, same namespace. Do NOT read that convergence as "both fields are @@ -234,6 +280,21 @@ "message", "note", "blocker", + # The deferred-work decision text a sweep parks on (`sweep.py`'s + # `decision-pending`) — operator-facing prose about the customer's own + # work, so it belongs beside `detail`/`reason`/`blocker`/`suggestion`/`note` + # under this set's free-text rule. Routed rather than left to `scrub_json` + # for the reason stated above that fallback: it collapses a MULTI-WORD + # question only by accident of `_IDENTIFIER_RE` forbidding spaces, and a + # one-token question (`"AcmeVault"`) is identifier-shaped and shipped + # verbatim (reproduced against `_scrub_entry`, 2026-08-30). No user-facing + # surface loses the text: both operator-facing readers take it from the RAW + # journal on the operator's own machine, not from this dump — + # `tui.data.pending_decision` (which returns the `(dw_id, question)` pair) + # and `tui.launch.decision_pending` (which answers the boolean). Named + # exactly, in both directions: an earlier version of this comment attributed + # `decision_pending` to `tui/data.py`, where no such symbol exists. + "question", "commit_message", "was_paused", "command", @@ -276,8 +337,75 @@ "stashed_to", } ) -# Journal fields whose value is a LIST of story keys (sprint unknown-keys). -_JOURNAL_KEYLIST_FIELDS = frozenset({"keys", "dw_ids"}) +# Journal fields whose value is a LIST of identifiers, aliased element-wise rather +# than dropped so a dump stays correlatable. Two namespaces live here: `keys` +# (sprint unknown-keys) and `story_keys` (`sweep._warn_stranded_bundles`, the +# bundle keys a cycle left in flight) are story keys; `dw_ids` are deferred-work +# ids. The fallback is what makes this routing necessary rather than cosmetic — +# `scrub_json` is the IDENTITY on a list of identifier-shaped strings, so an +# unrouted `story_keys` shipped its keys verbatim while the singular `story_key` +# beside it in the neighbouring record was aliased. +_JOURNAL_KEYLIST_FIELDS = frozenset({"keys", "dw_ids", "story_keys"}) + +# ``kind -> the field names that kind's record is DECLARED to carry``. On a kind +# listed here the usual ``scrub_json`` fallback is replaced by a fail-closed one: +# any key outside its declared set renders as ``_present`` rather than as a +# value. Every other routing rule above still runs first and still wins, so a +# declared-schema kind's ``story_key`` is aliased and its ``detail`` is dropped +# exactly as on any other kind — this table only decides what happens to the names +# nothing else claimed. +# +# It exists because ONE kind's field names are not authored by this codebase at all. +# ``engine._review_and_commit`` splats ``escalation.preference_escalations(rj)`` — +# entries lifted straight out of a session's own ``result.json`` — into +# ``journal.append``, so an LLM chooses the journal FIELD NAMES. The by-name tables +# cannot route a name nobody can enumerate, and the ``scrub_json`` fallback is the +# IDENTITY on an identifier-shaped scalar: an entry carrying +# ``customer="AcmeVault"`` came back byte-identical (reproduced against +# ``_scrub_entry``, 2026-08-30). +# +# The declared shape is ``{type, severity, detail}``. Its PROVENANCE, stated +# precisely because an earlier version of this comment got it wrong: the sole +# producer of this kind is ``engine._review_and_commit``, splatting a +# bmad-build-auto REVIEW session's ``result.json`` entries. Sweep does not journal +# ``preference-escalation`` at all. ``data/skills/bmad-loop-sweep/automation-mode.md`` +# declares the same three-key shape for a DIFFERENT session type, so it corroborates +# the shape and is not the contract that produces this record. Anything outside those +# three keys is off-schema and has no diagnostic claim on being shown verbatim. +# +# ⚠️ ACCEPTED RESIDUAL — decided 2026-08-30, not an oversight, and NOT to be +# re-filed or "fixed" as a fresh finding. This table closes the NAME axis only, and +# three things still reach the dump. Stated in full, because a disclosure that +# understates its own scope is the same defect as a comment naming the wrong +# mechanism. +# +# 1. The key NAME. ``AcmeVaultTenant=1`` renders as ``AcmeVaultTenant_present``. +# A name-free collapse (a single ``unrouted_field_count`` integer) closes this +# and was OFFERED AND DECLINED, in favour of the per-key marker's diagnostic +# value — a maintainer can see WHICH off-schema key a session invented, which is +# most of why the record is read. +# 2. Arbitrary key SHAPES, which follows from 1 and is easy to miss: nothing +# constrains an LLM-authored key to be identifier-shaped, so a free-text key +# survives as a JSON key with the suffix glued on — +# ``"customer AcmeVault owes 5k"`` renders as +# ``"customer AcmeVault owes 5k_present"`` (reproduced). +# 3. The VALUES of the three DECLARED names, which this table does not touch at +# all: they stay on the ``scrub_json`` fallback, so +# ``type="acmevault-tenant-isolation"`` and ``severity="AcmeVaultHigh"`` both come +# back byte-identical, as does a container in a declared name +# (``type={"customer": "AcmeVault"}``) — all reproduced. +# +# Do NOT "fix" 3 by routing ``type``/``severity``: the intent this table implements +# requires the declared schema to be emitted as it is today, on the ground that +# collapsing it destroys the field the record is read for. A further leak here is +# DISCLOSED — which is what this comment is — never routed. +# +# ``detail`` is named here for completeness even though ``_JOURNAL_DROP_FIELDS`` +# reaches it first: the set states the record's schema, and a schema that omitted a +# field because some other table happened to cover it would mislead the next reader. +_JOURNAL_KIND_SCHEMAS: dict[str, frozenset[str]] = { + "preference-escalation": frozenset({"type", "severity", "detail"}), +} # Policy keys whose values can carry secrets/paths/free text. Dropped or reduced # rather than scrubbed, since a single-token API key or repo name could be @@ -349,6 +477,11 @@ class TaskDiag: # dumps as `rearmed=True, attempt=1, n_sessions=2` — byte-identical to a HEALTHY # post-re-arm task. A counter, so it carries no customer content. generation: int + # DW-11's watermark: how far into the append-only `sessions` list an accepted + # resolution reached. Without it a support bundle cannot explain a SHORT + # `context.json` — a story whose older escalations are filtered out dumps + # identically to one that only ever raised the entries shown. A counter too. + escalations_resolved_upto: int dw_count: int n_sessions: int sessions: SessionTally @@ -630,6 +763,7 @@ def _task_diag(task: StoryTask, pseudo: sanitize.Pseudonymizer, weight: float) - spec_present=bool(task.spec_file), worktree_isolated=bool(task.worktree_path), generation=task.generation, + escalations_resolved_upto=task.escalations_resolved_upto, dw_count=len(task.dw_ids), n_sessions=len(task.sessions), sessions=_session_tally([task]), @@ -710,8 +844,14 @@ def _scrub_entry( first_ts: float | None, ) -> dict: """One journal entry reduced to a shareable form: relative timestamp, kind - verbatim, identifier fields aliased, free-text fields collapsed to a - presence boolean, and every remaining/unknown field scrub_json'd.""" + verbatim, identifier fields aliased, free-text fields collapsed to a presence + boolean, and every remaining/unknown field scrub_json'd. + + Two kinds of field never reach that last fallback, because for them + ``scrub_json`` fails closed only by accident of a value's shape. A name in + ``_JOURNAL_KEYLIST_FIELDS`` carrying something other than a list collapses to a + presence key, and on a kind with a declared schema + (``_JOURNAL_KIND_SCHEMAS``) so does every key the schema does not name.""" out: dict[str, Any] = {} ts = entry.get("ts") if isinstance(ts, (int, float)) and first_ts is not None: @@ -722,20 +862,55 @@ def _scrub_entry( # `looks_like_identifier` is not one of the three below anyway, and keying on the # placeholder would silently unroute every entry in a dump that had one. by_kind = _JOURNAL_KIND_ALIAS_FIELDS.get(kind, {}) + declared = _JOURNAL_KIND_SCHEMAS.get(kind) for k, v in entry.items(): if k in ("ts", "kind"): continue kind_ns = by_kind.get(k) if k in _JOURNAL_DROP_FIELDS: out[f"{k}_present"] = v is not None and v != "" - elif k in _JOURNAL_KEYLIST_FIELDS and isinstance(v, list): - ns = "story" if k == "keys" else "dw" - out[k] = [pseudo.alias(x, ns=ns, epic=epic_by_key.get(str(x))) for x in v] + elif k in _JOURNAL_KEYLIST_FIELDS: + if isinstance(v, list): + # Namespace by field, not by "everything that is not `keys`": both + # story-key list fields must land in the SAME namespace as the + # singular `story_key`, or one dump would carry two aliases for one + # story. + ns = "dw" if k == "dw_ids" else "story" + out[k] = [pseudo.alias(x, ns=ns, epic=epic_by_key.get(str(x))) for x in v] + else: + # A name in this set is DECLARED to carry a list of identifiers, so a + # non-list is an unknown shape — and falling through to `scrub_json` + # for it is exactly the accident this whole set exists to stop: a + # scalar `story_keys="1-1-acme-auth"` is identifier-shaped and ships + # verbatim (reproduced, 2026-08-30). Every producer passes a list + # today, so this is latent rather than live — which is the reason to + # fail closed on it rather than to rest on that survey staying true. + # A presence marker rather than an alias because the shape is + # genuinely unknown: a dict or an int has no sensible alias, and the + # one thing worth reporting is that the field was set. + out[f"{k}_present"] = v is not None and v != "" elif kind_ns is not None or k in _JOURNAL_ALIAS_FIELDS: ns = kind_ns or _JOURNAL_ALIAS_FIELDS[k] v = _alias_input(v, ns) epic = epic_by_key.get(str(v)) if ns == "story" else None out[k] = pseudo.alias(v, ns=ns, epic=epic) + elif declared is not None and k not in declared and k not in SELF_MINTED_FIELDS: + # A kind with a declared schema (`_JOURNAL_KIND_SCHEMAS`) replaces the + # `scrub_json` fallback with a fail-closed one, because on such a kind an + # unclaimed name is not a field nobody has routed yet — it is a field + # nobody in this codebase NAMED. See that table for the accepted residual: + # the key name itself still ships. + # + # `SELF_MINTED_FIELDS` is exempt because the premise does not hold for it. + # `Journal.append` stamps `log_task`/`log_pos` onto EVERY entry, including + # this kind's, so they are engine-authored and arrive on a record whose + # other keys are not. Collapsing them buys no safety — `log_task` is + # aliased before it ever reaches here, and `log_pos` is a byte offset — + # while `log_pos_present: true` would silently destroy the pane-log + # pointer on precisely the records an operator opens a dump to trace. + # Imported from `journal` rather than restated, so this exemption cannot + # drift from the `setdefault` pair that creates the fields. + out[f"{k}_present"] = v is not None and v != "" else: out[k] = sanitize.scrub_json(v) return out @@ -1045,15 +1220,20 @@ def render_markdown( # `gen` rides beside `att` because the pair is the discriminator: a # #705-class replay and a healthy post-re-arm task agree on every other # column here, so dropping it from the human report leaves the one field - # that separates them visible only under `--json`. + # that separates them visible only under `--json`. `esc-upto` rides beside + # `gen` on that same rule: DW-11's watermark is the only field separating + # "this story raised one escalation" from "its earlier ones are filtered + # out as already answered", and a short `context.json` is read off exactly + # this report. out.append( - "| alias | epic | phase | att | gen | rev | committed | spec | dw | sessions " - "| weighted | raw |" + "| alias | epic | phase | att | gen | esc-upto | rev | committed | spec | dw " + "| sessions | weighted | raw |" ) - out.append("|---|---|---|---|---|---|---|---|---|---|---|---|") + out.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|") for t in r.tasks: out.append( f"| `{t.alias}` | {t.epic} | {t.phase} | {t.attempt} | {t.generation} " + f"| {t.escalations_resolved_upto} " f"| {t.review_cycle} | {t.committed} | {t.spec_present} | {t.dw_count} " f"| {t.n_sessions} | {t.tokens.get('weighted', 0)} " f"| {t.tokens.get('total', 0)} |" diff --git a/src/bmad_loop/engine.py b/src/bmad_loop/engine.py index c43ded5f..8ea90b1b 100644 --- a/src/bmad_loop/engine.py +++ b/src/bmad_loop/engine.py @@ -40,7 +40,7 @@ session_failure_reason, ) from .install import dev_primitive_or_default -from .journal import Journal, save_state +from .journal import SELF_MINTED_FIELDS, Journal, save_state from .model import ( PAUSE_EPIC_BOUNDARY, PAUSE_ESCALATION, @@ -105,6 +105,16 @@ # read one immediate retry, but never dispatch another session over an unread # source: that later pass is allowed to replace the frontmatter list. HARVEST_REPAIR_READ_ATTEMPTS = 2 +# Journal field names a bound `journal.append(...)` call owns, and which therefore +# cannot be carried by a `**splat` of LLM-authored keys. `self`/`kind` are `append`'s +# bound parameters and `story_key` is already bound at the one splat site that needs +# this, so any of them arriving in the splat raises `TypeError: got multiple values +# for argument`; `ts` does not raise and instead silently overwrites the entry's real +# timestamp, since `append` builds `{"ts": now, "kind": kind, **fields}`. +# `log_task`/`log_pos` also do not raise: `append` stamps them with `setdefault`, so a +# supplied value silently wins and forges the pane-log pointer. Import the defining +# set from the minting site so this filter cannot drift from that pair. +_JOURNAL_RESERVED_KEYS = frozenset({"self", "kind", "story_key", "ts"}) | SELF_MINTED_FIELDS def _digest_of(text: str | None) -> str: @@ -2811,6 +2821,25 @@ def _review_and_commit( rj = result.result_json or {} for pref in preference_escalations(rj): + # `pref` is LLM-authored — it comes straight out of the session's own + # result.json — so its keys become journal field NAMES, and three of + # them collide with names this call already owns. `self` is bound by + # the method call and `kind`/`story_key` are bound above, so a + # result.json carrying any of them + # raised `TypeError: got multiple values for argument` and aborted + # the whole review leg over a field an agent invented (reproduced). + # `ts` does not raise and is worse for it: `Journal.append` builds + # `{"ts": now, "kind": kind, **fields}`, so a supplied `ts` silently + # OVERWRITES the real timestamp — `ts: 0` lands in the journal and + # every relative offset in a diagnostic dump is computed off it. + # Dropped rather than renamed: the record's declared schema is + # `{type, severity, detail}` (see `diagnostics._JOURNAL_KIND_SCHEMAS`), + # so a key spelled `kind`/`story_key`/`ts` is off-schema either way and + # would collapse to a presence marker in a dump regardless. + # `log_task`/`log_pos` also need filtering: `append` sets those with + # `setdefault`, so a caller value would silently replace its real + # pane-log pointer. All are journal-owned, not preference fields. + pref = {k: v for k, v in pref.items() if k not in _JOURNAL_RESERVED_KEYS} self.journal.append("preference-escalation", story_key=task.story_key, **pref) # A review pass is itself a bmad-build-auto run: it produces a spec # (status done/blocked + a refreshed followup_review_recommended), diff --git a/src/bmad_loop/frontmatter.py b/src/bmad_loop/frontmatter.py index cb4e3efd..fdbf888d 100644 --- a/src/bmad_loop/frontmatter.py +++ b/src/bmad_loop/frontmatter.py @@ -455,8 +455,11 @@ def set_frontmatter_status(path: Path, status: str, *, confine_root: Path) -> bo 46-byte spec to 12. The write is CONFINED, and this is the canonical statement of the rule the - three spec writers share (`verify.set_frontmatter_field` and - `devcontract._atomic_write_spec` restate it by reference): + FOUR writers of a spec's bytes share (`verify.set_frontmatter_field` and + `devcontract._atomic_write_spec` restate it by reference; `runs._restore_rearmed_spec` + — the re-arm transaction's UNDO — implements it so it can put back exactly what the + other three published, and a spec they can write but it refuses is the transaction's + write set going unhonoured): * A spec path under ``confine_root`` goes through `platform_util.atomic_write_bytes_confined`, which walks the components diff --git a/src/bmad_loop/journal.py b/src/bmad_loop/journal.py index 169517cf..4ef72b3b 100644 --- a/src/bmad_loop/journal.py +++ b/src/bmad_loop/journal.py @@ -24,6 +24,58 @@ # Verifier subprocess streams, deliberately NOT under LOGS_DIR — see # Journal.write_verify_stream for why sharing that directory is a TUI bug. VERIFY_DIR = "verify" +# The cycle-scoped artifacts a session writes into ``tasks//``: the ONE +# list the three sites that touch them share. Both adapters clear these in +# ``start_session`` (a caller-supplied task_id may be reused, and a silent session +# must not inherit a stale predecessor's outputs) and +# ``resolve._gather_escalations`` reads them back. Spelled here rather than three +# times, because a fourth artifact added to the reader alone would silently miss +# both adapters — which is the shape the parity was in before. +# +# ``result.json`` is the dev/review contract's own result file. ``escalation.json`` +# is the SWEEP SKILL's: its automation contract +# (``data/skills/bmad-loop-sweep/automation-mode.md``) tells a sweep session to +# write that file and then mirror the same entries into ``result.json``'s +# ``escalations``. That sentence lived in both adapters' comments and nowhere else, +# and it is the whole reason the reader opens two names rather than one. +# +# ORDER IS LOAD-BEARING — but NOT because of the mirroring, which is the obvious +# reading and the wrong one: ``_gather_escalations`` keys its map on canonical +# JSON, so a mirrored entry's STORED value is byte-identical whichever copy is read +# first. What the order fixes is the POSITION of DISTINCT entries in the +# newest-first list the operator is shown — result.json's entries precede +# escalation.json's, and a repeat keeps its first occurrence's slot. Swap these two +# and ``tests/test_resolve.py``'s +# ``test_gather_escalations_preserves_result_before_escalation_file_order`` and +# ``test_gather_escalations_keeps_a_duplicates_first_position`` redden (measured, +# not reasoned about). +# +# Appending a name is bounded twice, so it is not free. ``_gather_escalations`` +# JSON-parses every name here and skips anything that is not an +# ``{"escalations": [...]}`` document, so a name that does not carry that shape +# buys the reader nothing. And both adapters run this unlink loop AFTER +# ``start_session`` has already written ``prompt.txt`` into the same directory, so +# a name an earlier step of that method writes would be deleted on the way out. +# +# Four other cycle-scoped files live in ``tasks//`` and are deliberately +# NOT here, because each is owned and read by ONE adapter rather than shared: +# ``heartbeat.json``, ``resultless-stops.jsonl`` and ``session-lifecycle.jsonl`` +# (``adapters/generic.py``) and ``messages.json`` (``adapters/opencode_http.py``). +TASK_CYCLE_ARTIFACTS: tuple[str, ...] = ("result.json", "escalation.json") + +# The field names ``Journal.append`` stamps onto an entry ITSELF, rather than taking +# from its caller's keywords — see the ``setdefault`` pair in that method. No call +# site spells either one, which makes them invisible to anything reading call sites +# and easy for a consumer to mistake for a producer-supplied field. +# +# Spelled here, at the minting site, because two consumers need exactly this set and +# a third copy is how they drift: ``diagnostics._scrub_entry`` must exempt them from +# the fail-closed arm it applies to a declared-schema kind (they are engine-minted, +# never LLM-authored, so collapsing ``log_pos`` to a presence marker would throw away +# a byte offset for no safety gain), and ``tests/test_portability_guard.py`` needs +# them to keep its static call-site scan from calling them dead. Both import this +# name; neither restates the pair. +SELF_MINTED_FIELDS: frozenset[str] = frozenset({"log_task", "log_pos"}) class Journal: diff --git a/src/bmad_loop/model.py b/src/bmad_loop/model.py index 54923d36..d6b17f56 100644 --- a/src/bmad_loop/model.py +++ b/src/bmad_loop/model.py @@ -213,6 +213,19 @@ class StoryTask: # is deliberately NOT cleared when a task is reopened: the run-dir audit trail # it indexes is read by a later resolve cycle. generation: int = 0 + # How much of the append-only `sessions` list an accepted escalation resolution + # already covered: a LENGTH, i.e. an index INTO `task.sessions`, not a count of + # escalations and not a generation number. `resolve._gather_escalations` shows only + # the escalations recorded by sessions at or after this position, so a second + # resolve cycle does not re-present entries the human already disambiguated + # (DW-11). Stamped in `runs.rearm_escalation`, and only when its caller passes + # `resolution_recorded=True` — a re-arm that accepted nothing must not advance it, + # or escalations nobody answered become invisible forever. `record_session` is the + # sole mutation of `sessions` in `src/`, and a re-arm deliberately does NOT clear + # the list, which is what makes a length stable across cycles. 0 = nothing answered + # yet, which is also what a pre-upgrade `state.json` deserializes to (unfiltered, + # the pre-DW-11 behavior). + escalations_resolved_upto: int = 0 # set from the bmad-build-auto session's `followup_review_recommended` # frontmatter (PR #2505): when True and review.trigger = "recommended", the # orchestrator runs a follow-up review pass (bmad-build-auto re-invoked on the @@ -430,6 +443,7 @@ def to_dict(self) -> dict[str, Any]: "review_cycle": self.review_cycle, "followup_reviews_spent": self.followup_reviews_spent, "generation": self.generation, + "escalations_resolved_upto": self.escalations_resolved_upto, "followup_review_recommended": self.followup_review_recommended, "baseline_commit": self.baseline_commit, "baseline_untracked": self.baseline_untracked, @@ -598,6 +612,7 @@ def from_dict(cls, d: dict[str, Any]) -> "StoryTask": review_cycle=int(d.get("review_cycle", 0)), followup_reviews_spent=int(d.get("followup_reviews_spent", 0)), generation=int(d.get("generation", 0)), + escalations_resolved_upto=int(d.get("escalations_resolved_upto", 0)), followup_review_recommended=bool(d.get("followup_review_recommended", False)), baseline_commit=d.get("baseline_commit"), baseline_untracked=( diff --git a/src/bmad_loop/resolve.py b/src/bmad_loop/resolve.py index a5275da9..84458591 100644 --- a/src/bmad_loop/resolve.py +++ b/src/bmad_loop/resolve.py @@ -18,10 +18,13 @@ import os import subprocess from pathlib import Path +from stat import S_ISREG from typing import Any from .adapters.base import SessionSpec from .engine import _session_task_id +from .escalation import critical_escalations +from .journal import TASK_CYCLE_ARTIFACTS from .model import RunState from .platform_util import safe_segment from .runs import ( @@ -76,33 +79,198 @@ def read_resolution(run_dir: Path, story_key: str) -> dict[str, Any] | None: return doc -def _gather_escalations(run_dir: Path, state: RunState, story_key: str) -> list[dict[str, Any]]: - """The CRITICAL escalations recorded by this story's sessions, newest first. +def _gather_escalations( + run_dir: Path, + state: RunState, + story_key: str, + *, + start: int = 0, + skipped: set[str] | None = None, +) -> tuple[list[dict[str, Any]], int]: + """The CRITICAL escalations recorded by this story's sessions, newest first, + each DISTINCT escalation exactly once, paired with how many DISTINCT entries + were withheld as already answered. + + ``start`` is ``task.escalations_resolved_upto`` — a position in the append-only + ``task.sessions`` list, stamped by ``runs.rearm_escalation`` when a resolve cycle + recorded a resolution (DW-11). Records BELOW it were already put to the human and + answered, so their escalations are not shown again; the count of those the human + can no longer see is returned for the operator, never written into + ``context.json`` (the agent-facing contract is the unanswered set alone). The + default 0 reproduces the pre-DW-11 walk byte-for-byte, which is what a + pre-upgrade ``state.json`` deserializes to. + + Reads each session's tasks// artifacts — the same files the engine + inspected when it decided to pause. WHICH files is not spelled here: it is + ``journal.TASK_CYCLE_ARTIFACTS``, the one list this reader shares with the two + adapters that clear the same directory in ``start_session``, so a name added + there reaches all three sites at once. Ordering is `reversed(task.sessions)` + and, within a directory, that constant's own order — result.json before + escalation.json; a duplicate keeps its FIRST occurrence's position, which is + what preserves "newest first". Four guards, each for a defect this reader + hit on the way to the operator: + + * ``seen_ids`` — ``task.sessions`` is append-only and a re-arm deliberately + does NOT clear it, so state can carry two records under one ``task_id``. + ``sweep._rearm_generation`` bumps the id namespace for new restarts but + does not migrate records already persisted, and both records address the + SAME mutable ``tasks//escalation.json`` — reading it per record + attributes the abandoned cycle's escalation to the fresh session too. + Open each directory once. + * the content-keyed map — the sweep skill's own contract + (``data/skills/bmad-loop-sweep/automation-mode.md``) tells a producer to + write ``escalation.json`` and then mirror the same entries into + ``result.json`` ``escalations``. That mirroring is deliberate and stays; + the READER absorbs it, so a compliant producer is not shown to the human + twice. The key is canonical JSON because the two copies are parsed + separately — identity cannot see the mirroring and ``dict`` is unhashable + — and ``setdefault`` makes the first occurrence win. De-duplication is + global across the pass, not per directory; it removes only exact repeats, + so a directory holding CRITICAL A in one file and A + B in the other still + yields both. + * the ``except`` tuple and the ``list`` check — ``build_context`` is an + OBSERVATION path: a malformed artifact must cost its own contents and + nothing more, never raise out to the interactive resolve command. + ``UnicodeDecodeError`` is a ``ValueError``, not an ``OSError`` (the same + rationale recorded on ``read_resolution`` above), and ``json.loads`` can + also raise a plain ``ValueError`` when an integer exceeds Python's configured + digit limit. Deeply nested input can raise ``RecursionError`` while either + parsing the document or canonicalizing an entry, so both operations live + under the same artifact-level guard. Meanwhile, + ``critical_escalations`` iterates ``escalations`` with no list guard of its + own, so a ``{"escalations": null}`` artifact would raise ``TypeError`` + here. The guard belongs in this caller; the shared predicate stays the + single definition of CRITICAL. + * the ``stat`` classification and its ``S_ISREG`` check — deciding ABSENT from + UNREADABLE cannot go through ``Path.is_file()``, whose error behavior splits by + interpreter. Through 3.13 it re-raises anything outside + ``pathlib._IGNORED_ERRNOS`` (ENOENT, ENOTDIR, EBADF, ELOOP), so an artifact under + an EACCES directory raised straight out of this observation path and out of + ``cmd_resolve`` — the one thing the bullet above promises never happens. On 3.14 + the body became ``os.path.isfile``, which swallows every error and calls that + same artifact ABSENT, so nothing reached ``skipped`` and the caller stamped + coverage over escalations it never read. ``stat`` answers with an errno instead: + ENOENT and ENOTDIR are genuine absence and stay a bare ``continue`` — the + dominant case, since a task dir normally holds only one of these names, and + counting it as a skip would withhold coverage from every resolve cycle, + permanently. Everything else — EACCES, EIO, ESTALE, EBADF, and now ELOOP — is an + artifact that EXISTS and cannot be read, so it joins ``skipped`` under the same + shown-side condition. ELOOP changing sides is deliberate, and it is the one + reading this switch changes on EVERY interpreter rather than just one: a symlink + cycle answered False through 3.13 because ELOOP(40) is IN the ignored tuple, and + on 3.14 because ``os.path.isfile`` swallows it too. It is a degrade either way, + and this module withholds coverage rather than laundering one into a durable + claim. ``Path.stat`` is what makes a single ``except OSError`` enough here: + MEASURED as ``OSError`` errno 40 on 3.11, 3.13 and 3.14 alike, unlike + ``Path.resolve``, which raises ``RuntimeError`` on a loop under 3.11 and nothing + at all under 3.13. ``S_ISREG`` survives the switch because ``is_file()`` answered + False for a directory or a FIFO at that path while ``stat`` succeeds on both — + without it a directory would reach ``read_text`` as an ``IsADirectoryError``, and + a FIFO would BLOCK there forever, wedging the interactive command this reader + serves. + + The watermark is a FIFTH concern layered onto that same single walk, not a + second pass: ``reversed(task.sessions)`` reaches the unanswered tail first, so + entries are routed into two content-keyed maps by the record's own index and the + suppressed count is the answered keys that never appeared in the shown map. Two + consequences are deliberate. An entry raised on BOTH sides of the watermark is + shown and counted 0 — "not shown" is the claim the number makes, so it must never + count something the operator can see. And ``start`` only SELECTS a map; nothing is + indexed with it, so a watermark past the end of the list yields an empty shown + list rather than an IndexError. A ``task_id`` repeated across the watermark is + opened once by ``seen_ids``, at its newest occurrence — the shown side, the + conservative direction. - Reads each session's tasks//result.json (and escalation.json) — the - same files the engine inspected when it decided to pause.""" + ``skipped`` is an OUT-parameter, and it exists because the degrade above is + silent in exactly the place silence is unaffordable. An artifact dropped by the + ``except`` costs its own escalations — but ``runs.rearm_escalation`` then stamps + ``escalations_resolved_upto = len(task.sessions)``, which covers the session that + artifact belonged to. A transient read fault (a network mount, a truncated write + still in flight) therefore buries every escalation in it FOREVER: the next cycle + reads the file fine and withholds it as already answered. The caller refuses to + record coverage when this set is non-empty, which is the same + observe-and-degrade contract this reader already keeps — it just stops the + degrade from being laundered into a durable claim. + + It is an out-parameter rather than a third return value on purpose: the + ``(list, int)`` pair is asserted by ~20 tests and read positionally by + ``build_context``, and this signal has one consumer. Only skips on the SHOWN + side are recorded — ``target is found`` — because those are the records the + watermark would NEWLY cover. A skip below ``start`` was already covered by the + previous cycle's watermark, so re-covering it buries nothing; it costs only the + withheld COUNT, which claims nothing durable. Paths are collected rather than a + bare tally so a duplicate artifact cannot inflate the answer.""" task = state.tasks.get(story_key) - found: list[dict[str, Any]] = [] if task is None: - return found - for session in reversed(task.sessions): + return [], 0 + seen_ids: set[str] = set() + found: dict[str, dict[str, Any]] = {} + answered: dict[str, dict[str, Any]] = {} + last = len(task.sessions) - 1 + for offset, session in enumerate(reversed(task.sessions)): + if session.task_id in seen_ids: + continue + seen_ids.add(session.task_id) + target = found if last - offset >= start else answered task_dir = run_dir / "tasks" / session.task_id - for fname in ("result.json", "escalation.json"): + for fname in TASK_CYCLE_ARTIFACTS: fpath = task_dir / fname - if not fpath.is_file(): + try: + st = fpath.stat() + except (FileNotFoundError, NotADirectoryError): + continue + except OSError: + if skipped is not None and target is found: + skipped.add(str(fpath)) + continue + if not S_ISREG(st.st_mode): continue try: doc = json.loads(fpath.read_text(encoding="utf-8")) - except (OSError, json.JSONDecodeError): + if not isinstance(doc, dict): + raise ValueError("artifact is not a JSON object") + if "escalations" not in doc: + # The ordinary shape of a clean ``result.json``: nothing was + # raised, so there is nothing to show and nothing hidden. NOT a + # skip — counting it as one would withhold coverage from every + # resolve cycle, permanently. + continue + if not isinstance(doc["escalations"], list): + raise ValueError("'escalations' is not a list") + artifact_entries: dict[str, dict[str, Any]] = {} + for esc in critical_escalations(doc): + artifact_entries.setdefault(json.dumps(esc, sort_keys=True), esc) + except (OSError, ValueError, RecursionError): + if skipped is not None and target is found: + skipped.add(str(fpath)) continue - for esc in doc.get("escalations", []) if isinstance(doc, dict) else []: - if isinstance(esc, dict) and str(esc.get("severity", "")).upper() == "CRITICAL": - found.append(esc) - return found + for key, esc in artifact_entries.items(): + target.setdefault(key, esc) + return list(found.values()), sum(1 for key in answered if key not in found) -def build_context(state: RunState, run_dir: Path, story_key: str, *, isolation: str) -> Path: - """Write resolve//context.json for the resolve skill to read. +def build_context( + state: RunState, run_dir: Path, story_key: str, *, isolation: str +) -> tuple[Path, int, int]: + """Write resolve//context.json for the resolve skill to read, and + return it beside the number of already-answered escalations withheld from it and + the number of session artifacts this walk could NOT read. + + The withheld count is for the OPERATOR's terminal (`cli.cmd_resolve` prints it) and + is deliberately not a `context.json` field: the skill's contract is singular — + resolve the escalation you are shown — and a count of things the agent cannot see is + not something it can act on. It comes from the same single walk that produced the + shown list, never from a second `_gather_escalations` call subtracting lengths. + + The unreadable count rides the SAME walk for the same reason, and it is a third + return value rather than a second out-parameter because it has to cross a process's + worth of control flow: `cli.cmd_resolve` is the only surface that can advance + `escalations_resolved_upto` (the TUI hard-codes `resolution_recorded=False`), and a + non-zero count there means this cycle showed the human strictly less than the + watermark would claim they answered. The caller withholds coverage on it — see + `_gather_escalations`' `skipped` for why the alternative is permanent burial. Zero + on every ordinary run, so the coverage path is unchanged whenever the run-dir reads + cleanly. `isolation` is the LIVE policy's `scm.isolation`, and it is required rather than defaulted for the reason this surface exists at all: three of the fields below — @@ -131,6 +299,17 @@ def build_context(state: RunState, run_dir: Path, story_key: str, *, isolation: # the main checkout while `stories_engine._stories_folder` was still the mount, so # one `context.json` could name two trees. stories_root = task_stories_root(task, state) + # DW-11: hide what an earlier resolve cycle already answered. `start` is the task's + # own watermark — 0 for a task never resolved, and for every pre-upgrade + # `state.json`, which is the unfiltered pre-DW-11 walk. + unreadable: set[str] = set() + escalations, withheld = _gather_escalations( + run_dir, + state, + story_key, + start=task.escalations_resolved_upto if task else 0, + skipped=unreadable, + ) context = { "story_key": story_key, "run_id": state.run_id, @@ -151,7 +330,7 @@ def build_context(state: RunState, run_dir: Path, story_key: str, *, isolation: "spec_file": (task_spec_path(task, state).as_posix() if task and task.spec_file else None), "baseline_commit": task.baseline_commit if task else None, "paused_reason": state.paused_reason, - "escalations": _gather_escalations(run_dir, state, story_key), + "escalations": escalations, # as_posix so the context contract is the same string on every OS (the # path is consumed by the agent, and Python/tools accept '/' on Windows). "resolution_path": resolution_path(run_dir, story_key).as_posix(), @@ -204,7 +383,7 @@ def build_context(state: RunState, run_dir: Path, story_key: str, *, isolation: path = context_path(run_dir, story_key) path.parent.mkdir(parents=True, exist_ok=True) path.write_text(json.dumps(context, indent=2), encoding="utf-8") - return path + return path, withheld, len(unreadable) def _stories_context(state: RunState, story_key: str, root: Path) -> dict[str, Any]: diff --git a/src/bmad_loop/runs.py b/src/bmad_loop/runs.py index 21b251ae..388832fe 100644 --- a/src/bmad_loop/runs.py +++ b/src/bmad_loop/runs.py @@ -35,6 +35,7 @@ UnconfinedWriteError, _mkstemp_beside, atomic_replace, + atomic_write_bytes, atomic_write_bytes_confined, atomic_write_text_confined, create_exclusive_confined, @@ -3040,20 +3041,20 @@ def task_spec_root(task: StoryTask, state: RunState) -> Path: shape: `model._serialized_worktree_path` keeps a path verbatim exactly when `relative_to(worktree_path)` raises, so the two spellings did not share a prefix. Returning the worktree there would name a root that can never contain the path - `task_spec_path` passes through — the three `_atomic_write_spec` writers would - silently take the plain no-follow arm (losing #593's O_NOFOLLOW walk) and - `_restore_rearmed_spec`, which calls the confined writer directly, would RAISE. - - The project can often confine it. Where nothing can, the THREE `_atomic_write_spec` - writers land on the arm they already took — they select lexically, so an out-of-root - path simply takes the plain no-follow write as before. That is not true of every - writer: `_restore_rearmed_spec` calls `atomic_write_bytes_confined` DIRECTLY with no - lexical arm, so for a spec outside both the mount and the project — the shared + `task_spec_path` passes through — all FOUR writers of these bytes would silently take + the plain no-follow arm and lose #593's O_NOFOLLOW walk. + + The project can often confine it. Where nothing can, every writer of these bytes + lands on the arm it already took — they all select LEXICALLY, so an out-of-root path + simply takes the plain no-follow write as before. That parity is load-bearing and was + once broken: `_restore_rearmed_spec` called `atomic_write_bytes_confined` DIRECTLY + with no lexical arm, so for a spec outside both the mount and the project — the shared artifact dir `_spec_is_shared_with_the_redrive` treats as first-class and reachable — - it raises `UnconfinedWriteError` and the re-arm's undo is lost with the spec already - flipped and stripped. That asymmetry PRE-DATES this anchor (the previous body - returned the worktree there, which equally cannot confine the path) and is tracked - separately; it is named here so the paragraph is not read as covering it. + the flip, the strip and the re-stamp all LANDED while the re-arm's undo alone raised + `UnconfinedWriteError`, losing the rollback on precisely the specs it could still + break. A writer that refuses where its siblings write does not add safety here; it + subtracts the transaction. Do not re-introduce the asymmetry by "hardening" one of + the four in isolation. The arm is not unconditionally an improvement either, and that exception is graded by `test_task_spec_root_refuses_a_spec_the_project_cannot_reach`: `_atomic_write_spec` @@ -3482,62 +3483,283 @@ def _redrive_reads_the_upstream_artifacts(state: RunState) -> bool: def _restore_rearmed_spec( spec_path: Path, original: bytes | None, task: StoryTask, state: RunState -) -> None: - """Put back the bytes a re-arm FOUND on the spec, for the aborts that can fire after - a write has already landed. +) -> Literal["restored", "unchanged", "unknown"]: + """Put back the bytes a re-arm FOUND on the spec, and say what is now on disk. `rearm_escalation` holds an invariant its own refusals depend on: an aborted re-arm leaves the spec byte-identical, so the escalation stays armed and the human can fix - the file and re-run resolve. TWO of its four refusals earn that by SEQUENCING alone — - the flip's read-back check and the `FrontmatterWriteError` arm both raise before - `devcontract.strip_auto_run_result` runs, which is why that strip is deliberately - ordered after them, and `set_frontmatter_status` decides it cannot move a `status:` - before it writes anything. The other two cannot be sequenced out of the hazard, and - both call this: - - * The baseline re-stamp needs `task.baseline_commit` from the advance, and the - advance must itself run after the spec block (a just-cleared stories sentinel would - otherwise be captured into `baseline_untracked` as phantom pre-existing residue). - * The `(OSError, UnicodeDecodeError)` arm spans BOTH spec helpers, and the strip is - the later one — a fault raised inside it is raised after the flip published. - - By the time either can fail, the status flip has landed and `save_state` has not — so - the abort would otherwise leave the run's task ESCALATED against a spec already - flipped to the re-drive's status and (for the re-stamp) stripped of the terminal - `## Auto Run Result` the next resolve session reads as its context. That is exactly - the "one edit nothing else records" the sequencing exists to prevent. - - Writes only what it can prove it changed. `original` is `None` when the spec was - unreadable before the first write (there is then nothing to restore, and nothing - could have been written either), and a spec that is gone or unreadable NOW is not a - state this undo can improve — recreating a file another process removed would fight - a concurrent actor rather than restore this function's own edit. Bytes equal to - `original` mean nothing landed, so nothing is rewritten and the mtime is left alone. - - Byte-verbatim and CONFINED, matching the writes it undoes: `atomic_write_text_confined` - would re-encode and translate newlines, so a CRLF spec would come back subtly - different from the file this re-arm found, and an unconfined write would drop the - `O_NOFOLLOW` walk of the parent components (#593) that every other write to this path - takes. A restore that itself fails RAISES rather than degrading — the spec is then + the file and re-run resolve. That used to be earned twice over — by SEQUENCING for + the two refusals that raise before `devcontract.strip_auto_run_result` runs (which is + why that strip is still deliberately ordered after the flip's read-back check), and + by two hand-placed calls to this function for the two that could not be sequenced out + of the hazard. Neither half covered the rest of the window: a `journal.append` + OSError from the residue pass, a non-Git fault from the commits probe, or a failing + `save_state` each escaped with the flip published and the task still ESCALATED. + + So there is now ONE caller, `_rollback_rearm`, invoked from the transaction guard + that spans the whole window from the first spec write to `save_state`. The sequencing + is not redundant — it is what keeps those two refusals from ever writing in the first + place — but the undo no longer depends on someone remembering to place it. + + THREE outcomes rather than a bool, because the operator surfaces make a CLAIM about + the file and only one of the non-restoring cases entitles them to it: + + * ``"restored"`` — a write had landed and was put back. + * ``"unchanged"`` — the file was READ and PROVED byte-equal to `original`, so nothing + landed, nothing is rewritten, and the mtime is left alone. This is the only answer + that licenses "the spec was left exactly as the re-arm found it". + * ``"unknown"`` — this undo cannot say. Two shapes reach it. `original` is `None`, + meaning the spec was unreadable at capture time or the re-arm never entered the + spec block at all (the sentinel-clear leg, which has already UNLINKED the file — + so a reader told "unchanged" there would be told a deleted file proves the tree is + untouched). Or the spec is gone or unreadable NOW, in which case the undo could not + even look: recreating a file another process removed would fight a concurrent actor + rather than restore this function's own edit, so it declines — but declining is not + the same as proving nothing landed. + + Folding those into one "nothing had to be put back" answer is what made the notice + overclaim, so the distinction lives in the return type rather than in a comment. + + Byte-verbatim, never the text writer: `atomic_write_text_confined` would re-encode and + translate newlines, so a CRLF spec would come back subtly different from the file this + re-arm found. + + And it picks its arm the SAME LEXICAL WAY the three writers it undoes do + (`frontmatter.set_frontmatter_status` states the rule; `verify.set_frontmatter_field` + and `devcontract._atomic_write_spec` restate it): under `confine_root`, through the + component-walking confined helper (#593); outside it, the plain `follow_symlinks=False` + write. Calling the confined helper unconditionally looked stricter and was strictly + worse — an artifacts folder configured OUTSIDE both the mount and the project is + supported configuration (`bmadconfig` resolves one, `verify.spec_within_roots` trusts + it, `_spec_is_shared_with_the_redrive` treats it as first-class), and there the flip, + the strip and the re-stamp all LAND while this undo alone raised + `UnconfinedWriteError`. The undo then reported `failed` on exactly the specs it was + able to break, which is the asymmetry `task_spec_root`'s docstring used to name as + out of scope. A restore that refuses where the writes succeeded is not extra safety; + it is the transaction's write set going unhonoured. + + A restore that itself fails RAISES rather than degrading — the spec is then half-written and only the operator can settle it, which is the loudest thing this can - be. `UnconfinedWriteError` is an `OSError`, so the one arm covers both. + be. `UnconfinedWriteError` is an `OSError`, so the one arm still covers both. + + `unknown` is reserved for the two shapes that are ANSWERS rather than failures to + look: nothing was captured, and the spec is gone. A read that merely could not be + performed is neither, and must not short-circuit the undo — see the `except` arms. """ if original is None: - return + return "unknown" try: if spec_path.read_bytes() == original: - return + return "unchanged" + except FileNotFoundError: + # The one read fault that is an ANSWER about the disk: the spec is gone, so there + # are no bytes carrying this re-arm's flip and nothing for the undo to put back. + return "unknown" except OSError: - return + # Every other read fault (EIO, EMFILE, a transient EACCES) says nothing about + # what is ON DISK — and this read is only the "already identical, skip the write" + # shortcut. Answering `unknown` here abandoned the restore on exactly the runs + # that still needed it: the spec keeps the flip and the stripped result section + # while `save_state` leaves the story ESCALATED, which is the split state this + # whole transaction exists to prevent. Fall through and attempt the write; it + # raises `RearmError` if it cannot land, which is the loud outcome the docstring + # above promises. The cost of being wrong here is one redundant identical write. + pass + confine_root = task_spec_root(task, state) try: - atomic_write_bytes_confined(spec_path, original, confine_root=task_spec_root(task, state)) + if spec_path.is_relative_to(confine_root): + atomic_write_bytes_confined( + spec_path, original, confine_root=confine_root, require_writable_target=True + ) + else: + atomic_write_bytes( + spec_path, original, follow_symlinks=False, require_writable_target=True + ) except OSError as e: raise RearmError( f"cannot restore {spec_path} after a failed re-arm " - f"({e.__class__.__name__}: {e}) — the spec carries this re-arm's status flip " - "and has lost its `## Auto Run Result` section, while the story is still " - "escalated; restore the spec from git, then re-run resolve" + f"({e.__class__.__name__}: {e}) — the spec may carry this re-arm's status " + "flip and may have lost its `## Auto Run Result` section, while the story is " + "still escalated; restore the spec from git or from your own copy, then " + "re-run resolve" ) from e + return "restored" + + +def _rollback_rearm( + journal: Journal, + story_key: str, + spec_path: Path | None, + spec_before: bytes | None, + task: StoryTask, + state: RunState, + error: BaseException, +) -> None: + """Undo an aborted re-arm's spec writes and RECORD that the re-arm aborted. + + The error-path half of `rearm_escalation`'s transaction. Its caller re-raises the + original fault immediately after, so nothing here may return a verdict or swallow + one: this function's whole job is to leave the spec's bytes as the re-arm found them + and put the fact on the run's audit trail. + + `rollback` goes ON the record, not left to be re-derived, because every reader is + OUT of process: `rearm_event_notice` renders from a journal line alone, with neither + the task nor the tree to consult, and the outcomes need different sentences. + `restored` and `unchanged` are `_restore_rearmed_spec`'s own answers and both mean + the spec on disk is what the re-arm found. `failed` means the restore itself could + not write, the one outcome that can leave a HALF-WRITTEN spec — recorded from a + `finally` for exactly that reason, since that arm re-raises. `unknown` means no such + claim is available: either this re-arm never resolved a spec path at all, or the undo + could not read the file to prove anything. + + The sentinel-clear leg lands in `unknown` and that is the point. It runs INSIDE the + guarded window but writes no spec bytes — it UNLINKS the sentinel — and the + transaction deliberately does not undo it (`_clear_sentinel` preserves a copy under + `{run_dir}/sentinels/` and a retried resolve re-clears it idempotently). What the + transaction covers is the spec's BYTES from the first spec write onward; the deletion + is outside that, so the record must not claim the tree is as the re-arm found it. + + The record is journalled through a `finally` and every `Exception` from that append + is suppressed. Recording an abort is an OBSERVATION, and an observation that cannot + be made must not replace the fault the operator is being told about — while a restore + failure is a repair write, and repair writes raise. + + `Exception` and not `OSError`, which is the ONE place in this transaction where the + breadth is deliberately NARROWER than the guard's own `BaseException` and, at the + same time, wider than a filesystem taxonomy. Wider, because `Journal.append` + serializes caller-supplied values and opens a file: a `TypeError` or `ValueError` out + of `json.dumps`, or anything else this append can raise, would otherwise REPLACE the + fault the whole record exists to report — the `Always:` re-raise invariant the two + pinned `MemoryError` tests depend on. Narrower, because `KeyboardInterrupt` and + `SystemExit` must still leave: by the time this `finally` runs the rollback has + already completed, so an interrupt here cannot reproduce DW-79/DW-83, and discarding + the operator's Ctrl-C to keep a breadcrumb would be the worse trade. + """ + # `unknown` is the floor, not `unchanged`: a re-arm that resolved no spec path made + # no claim about any file, and the surfaces must not manufacture one for it. + rollback = "unknown" + try: + if spec_path is not None: + rollback = _restore_rearmed_spec(spec_path, spec_before, task, state) + except BaseException: + rollback = "failed" + raise + finally: + try: + journal.append( + "rearm-aborted", + story_key=story_key, + # `""` rather than the key again when the re-arm never resolved a spec + # path: the field is the SPEC's locator on every other `rearm-*` kind, + # and a reader that finds a story key there would alias it into the wrong + # namespace and render it as a spec that does not exist. + spec_file=str(spec_path) if spec_path is not None else "", + error=f"{error.__class__.__name__}: {error}", + rollback=rollback, + ) + except Exception: # nosec B110 - the OBSERVATION must not replace the fault + # See the docstring: the ORIGINAL re-arm fault wins over ANY ordinary + # failure of this append, not just a filesystem one. `KeyboardInterrupt` + # and `SystemExit` are not `Exception` and still propagate. + pass + + +def _rearm_commit_landed(run_dir: Path, story_key: str, task: StoryTask) -> bool: + """Did `save_state` already COMMIT this re-arm, despite the fault now unwinding? + + `journal.save_state` ends in `atomic_replace`, so the commit is a single rename that + either happened or did not — but the CALL can still fail after it: a `KeyboardInterrupt` + delivered between that rename and the return unwinds through the transaction guard + with `state.json` already describing a PENDING, re-armed task. Rolling the spec back + there does not restore the pre-re-arm world; it MANUFACTURES the mirror image of + DW-79/DW-83 — persisted state re-armed against a spec that is not — and then reports + it as "nothing was persisted, the story is still escalated", which is simply false. + + The guard cannot know this from control flow (no assignment after `save_state` runs + on that path), so it ASKS THE DISK, which is the only witness of a rename. Both the + bumped `generation` and the reset `phase` must match the object `save_state` was + handed: `generation` alone would be satisfied by a state file this call never wrote + only if some other writer had minted the same bump, and `phase` alone moves for + reasons a re-arm does not own. + + Those two conjuncts are a sufficient identity ONLY because `rearm_escalation` runs as + the SOLE writer of this run's `state.json`, and that model is the probe's premise + rather than an assumption left implicit. Exactly TWO call sites reach this + transaction — `cli.cmd_resolve` and `tui.TuiApp._do_rearm` — and each consults + liveness before any side effect: :func:`engine_liveness` in the CLI, its pid-file + sibling :func:`liveness` in the TUI (`probe_liveness` is the shared body). A third + control command, `cli.cmd_resume`, never re-arms but DOES write this run's + `state.json` (through `_resume_paused_run`), which is why the sole-writer claim has + to account for it as well as for the two callers. + `tests/test_portability_guard.py::test_rearm_escalation_called_only_behind_a_liveness_gate` + holds that enumeration, which is otherwise prose a third call site could falsify + silently. + + Those gates establish that no engine is PROVABLY ALIVE — not that one is proven + dead — and the premise rests on the difference, so it is stated rather than rounded + off. `"alive"` is refused outright at all three. `"unknown"` is not: `cmd_resolve` + proceeds on it under `--force`, `cmd_resume` warns and proceeds by design (it is the + recovery path that rewrites engine.pid), and the TUI counts it as blocking only for a + pid-backed run. So the model this probe leans on is the engine stopped AND the + operator driving one control command at a time. Under it only THIS caller can have + moved either field, which is exactly what the exact-phase predicate reports — the + predicate is correct for the reason it is narrow. + + Two overlapping control commands are OUTSIDE that model rather than handled by it, + and deliberately so. `journal.save_state` stages through a FIXED `state.json.tmp` + sibling before its `atomic_replace` — the collision `_write_stop_request` documents + under #379, which names the stop-request file as the ONE control file with genuinely + *concurrent* writers — so two overlapping re-arms lose a `save_state` to + `FileNotFoundError` long before this probe's identity could matter. Answering them + here was weighed and declined: a lock taken by only `rearm_escalation` excludes + nobody (the honest fix is a run-level one shared with `_resume_paused_run` and the + engine's own `save_state`), and a durable per-re-arm token stamped on `StoryTask` + would buy this probe a precision the `save_state` writer beneath it cannot honour, at + the cost of a new persisted model field. Tracked as DW-93; the probe stays two + conjuncts over the reloaded task. + + Degrades to `False` — roll back, the pre-existing behavior — on ANY failure to read + or parse the state file. This is observation feeding a repair decision, and the safe + default is the one that leaves the spec as the re-arm found it: a re-arm that did + NOT commit and is wrongly believed to have is the DW-79/DW-83 defect itself, while + the converse leaves a rolled-back spec beside committed state that the next resume + re-drives from a spec still carrying the escalated status — recoverable, and loud. + + ANY failure means `BaseException`, and that breadth is the whole reason this probe is + safe to call where it is called. Its ONE call site sits inside the transaction guard's + `except BaseException` arm and runs BEFORE `_rollback_rearm`, so a fault escaping this + function escapes the guard too and the rollback never happens — leaving exactly the + spec-flipped-against-an-ESCALATED-task state the guard exists to end, now reached by + the code added to prevent its mirror image. `load_state` reads and parses a file, so a + `KeyboardInterrupt` or `SystemExit` delivered anywhere in it is not hypothetical, and + under `except Exception` it took precisely that path. + + Swallowing an interrupt here is therefore the correct trade, and it is not a lost + Ctrl-C: the rollback is a REPAIR WRITE that must not be skipped, and the guard's own + `raise` still propagates the original re-arm fault immediately afterwards, so the + process still exits loudly — one spec-sized write later. This is the reverse of the + trade `_rollback_rearm`'s abort-record append makes, and the two are consistent + because the acts differ: recording is an observation and must never displace a fault, + while repairing is a write whose omission IS the defect. The abort record's append + remains the ONE place in this transaction whose breadth is narrower than the guard's. + + No `rearm-aborted` record is written on the committed path either (the caller skips + the whole rollback). Every rendering of that kind asserts that nothing was persisted; + there is no value of `rollback` that is true here, and inventing one would put a + false sentence on both operator surfaces rather than leave the fault to speak. + """ + try: + persisted = load_state(run_dir).tasks.get(story_key) + except BaseException: + # See the docstring: a fault escaping this probe escapes the guard arm that + # calls it and skips the rollback entirely, so an interrupt is absorbed here + # and the original fault still propagates from the guard's `raise` below. + return False + return ( + persisted is not None + and persisted.generation == task.generation + and persisted.phase == task.phase + ) def _redrive_spec_status(state: RunState, task: StoryTask, *, isolated_redrive: bool) -> str: @@ -3657,6 +3879,7 @@ def rearm_escalation( *, restore_patch: str | None = None, isolated_redrive: bool, + resolution_recorded: bool, ) -> str: """Re-arm an escalation-paused story so the next resume re-drives it. @@ -3680,7 +3903,10 @@ def rearm_escalation( otherwise let the re-drive re-mint a session id byte-equal to one the abandoned attempt already recorded (#705). `task.sessions` is deliberately NOT cleared — a second resolve cycle reads that run-dir audit trail — so - the id is what has to change. + the id is what has to change. That preserved trail is also what + `resolution_recorded` watermarks: keeping it whole is what lets a later + cycle tell the answered prefix from the unanswered tail, instead of + choosing between re-presenting everything and losing the audit (DW-11). - The spec's `baseline_revision` is re-stamped on BOTH legs, and only when the advance above actually RAN — `advanced` records that both git reads succeeded, not that HEAD changed, so a resolve session that committed nothing still @@ -3723,6 +3949,25 @@ def rearm_escalation( defect this parameter exists to close. Both callers (`cli.cmd_resolve`, `tui.TuiApp._do_rearm`) hold a loaded policy already. + `resolution_recorded` says whether THIS gesture accepted a resolution, and it + alone gates the `escalations_resolved_upto` watermark (DW-11): the next resolve + cycle hides every escalation recorded below it, so advancing it over entries no + human answered would bury them forever and report them as already answered — the + inverse of the defect the watermark exists to fix. Keyword-only and REQUIRED for + the same reason as `isolated_redrive`: a default would be wrong in silence on + exactly the path that matters. It is a PARAMETER rather than a disk read because + the fact is not on disk. `resolution.json` is unlinked at one site in `src/` + (`resolve.run_session`, before it launches), which only `cli.cmd_resolve`'s + interactive arm reaches, and nothing deletes the marker at or after a re-arm — so + the marker survives the re-arm that consumed it, and `resolve --no-interactive` or + the TUI's Re-arm button would read the PREVIOUS cycle's marker as its own. The + caller already holds the answer: `cmd_resolve` binds it from `resolve.run_session`, + and both non-interactive callers know by construction that no session ran. Do not + unlink the marker here either — the TUI's Re-arm button is gated on its presence. + + The generation bump stays UNCONDITIONAL beside the gated stamp: it answers session-id + reuse (#705), which an abandoned attempt needs exactly as much as a resolved one. + Returns the re-armed story key. Raises RearmError when the run is not paused at the escalation stage, the target story is not escalated, or a supplied `restore_patch` fails `validate_restore_latch` (the shared precondition set — @@ -3769,6 +4014,17 @@ def rearm_escalation( # replay the abandoned verdict for the fresh attempt (#705). Bumped BEFORE any # dispatch, so the id is unique from the re-drive's first session onward. task.generation += 1 + # DW-11. How much of the preserved audit trail this resolution covered, so the next + # `resolve` shows the human only what they have not already answered. Gated on the + # CALLER's answer, never on `resolution.json`: the marker survives the re-arm that + # consumed it (only `resolve.run_session` unlinks it, and two of the three callers + # never run one), so reading it here would let a later marker-less gesture stamp + # over escalations nobody saw. A length, taken BEFORE the re-drive appends anything + # — `record_session` is the sole mutation of this list — and left where it stands + # when nothing was accepted, which reproduces the pre-DW-11 behavior for that + # gesture: everything shown, nothing reported withheld. + if resolution_recorded: + task.escalations_resolved_upto = len(task.sessions) task.review_cycle = 0 task.followup_reviews_spent = 0 # human-resolved re-drive gets a fresh damping budget task.defer_reason = None @@ -3778,550 +4034,635 @@ def rearm_escalation( # a prior restore attempt the human then chose to redo from scratch. task.restore_patch = restore_patch - # The bytes this re-arm found on the spec, for `_restore_rearmed_spec`. Declared out - # here because the baseline re-stamp that consumes it sits in a SECOND - # `if task.spec_file:` block, past the advance it depends on. + # The spec this re-arm writes to and the bytes it FOUND there — the two inputs the + # rollback below needs. Declared out here because their consumers sit past every + # block that sets them: the baseline re-stamp's SECOND `if task.spec_file:` block, + # and the transaction guard's `except` arm, which has to name them from outside all + # of them. spec_before: bytes | None = None - if task.spec_file: - spec_path = task_spec_path(task, state) - # Stories mode only: a fixed-slug pre-planning-halt sentinel - # (`-unresolved.md` / `-ambiguous.md`) is cleared by deletion, not a - # status flip. Clear it ONLY when the run recorded this task AS a sentinel at - # detection time (`task.sentinel_kind`, stamped by StoriesEngine's pick-time - # wedge / post-dev read-back) — never by re-deriving from the basename. That - # keeps a real story spec that merely happens to be named `-unresolved.md`, - # or a *non-sentinel* escalation whose spec matches the convention, on the - # status-flip path so it is kept, not deleted. Gate on the run source too (the - # convention exists only in stories mode) and defensively re-confirm the - # on-disk name still matches the recorded slug before deleting. - sentinel_kind = task.sentinel_kind if state.source == "stories" else "" - if sentinel_kind and _sentinel_condition(spec_path, key) == sentinel_kind: - # a sentinel is cleared by deletion, not a status flip; drop the stale - # spec_file so the re-dispatch starts from PENDING (clean re-plan). - _clear_sentinel(run_dir, journal, spec_path, key, sentinel_kind) - task.spec_file = None - task.sentinel_kind = "" # verdict discharged; the re-dispatch is clean - # Deleting the sentinel does not make the re-plan produce a different one: - # the correction that does lives UPSTREAM, in the `SPEC.md` / `stories.yaml` - # the resolve skill sends the agent to instead of this file. That correction - # faces the same reachability gap the spec arm below measures, and faced NO - # gate at all — this arm cleared `spec_file` and fell through, so - # `write_reaches_the_redrive` was never computed and the resume was never - # held for a sentinel. An isolated re-drive then mounts fresh from - # `redrive_base_ref`, re-plans from a committed tree that never saw the - # edit, mints the same sentinel again, and the escalation is spent. - # - # Narrowed by PROOF for the reason the spec record below is, and the need is - # sharper here: `stories_reach_the_redrive` answers "unreachable" for EVERY - # isolated stories run whose spec folder sits inside the project, which is - # every one we author. Gating on it alone would fire — and hold the resume — - # on 100% of isolated sentinel re-arms, a per-configuration constant rather - # than an event. `_redrive_reads_the_upstream_artifacts` is what makes it an - # event: it fires only while this checkout still holds upstream bytes the - # ref the re-drive mounts from does not. - # - # No `redrive` discriminator, unlike the spec record: this one has a single - # remedy because it has a single reachable shape. An in-place re-drive reads - # the main checkout's working tree, which is exactly where `cwd=project` put - # the correction, so `stories_reach_the_redrive` short-circuits that leg to - # reachable and no record is written for it at all. - if not stories_reach_the_redrive( - task, state, isolated_redrive=isolated_redrive - ) and not _redrive_reads_the_upstream_artifacts(state): - journal.append( - "rearm-upstream-write-unreachable", - story_key=key, - # `task_stories_root` names the tree the RUN owns; the correction - # lands in the checkout the resolve session ran in. Both are the - # project on this leg unless a mount is recorded, and the operator - # needs the folder to act, so the record carries the folder the - # remedy is about rather than the run's read locator. - stories_root=str(_upstream_artifacts_folder(state)), - target_branch=state.target_branch, - ) - else: - # A WORKTREE-LOCAL spec's writes below land in the unit's worktree - # (`task_spec_path`) — which the re-drive destroys before reading anything. - # A re-armed task (phase PENDING, `defer_reason` cleared, and no resumable - # session because `generation` was just bumped) falls to - # `engine._finish_inflight`'s final arm, which calls `discard_worktree` and - # lets `_run_story` mount a fresh one. The re-driven session then resolves - # its spec through `verify.resolve_spec_path(task.spec_file, - # workspace.paths)` (`engine._dispatched_spec_for_attempt`), and under - # isolation `workspace.paths` is rebased onto that FRESH worktree, which - # checks out TRACKED files only. So the re-drive reads the COMMITTED spec. - # - # No working-tree write reaches it — not this one, and not a write to the - # main checkout either: the fresh worktree comes from git rather than from a - # copy of that tree, and `seed_adapter_defaults` seeds adapter config files, - # not the output folder. The channel that DOES work is the human committing - # the corrected spec from the resolve session, which runs with `cwd=project`. - # The writes below are kept (they are correct for the in-place case, and - # harmless here), but the operator is told — a flip that cannot land is - # exactly the silent re-wedge #640(b) exists to end. - # - # "Worktree-local" is the load-bearing qualifier, and isolation does not - # imply it: an artifact dir configured OUTSIDE the project tree is shared - # across checkouts by `ProjectPaths.rebased`, so a spec that landed there is - # one file the fresh worktree reads through the very absolute path this - # writes to. `_spec_is_shared_with_the_redrive` carves out that case, and only - # that one: the main checkout's copy is outside the worktree too, and stays - # unreachable because the re-drive measures it against worktree-local roots. - # Route /bmad-build-auto via the spec's frontmatter status (decision - # table): patch-restore -> in-review -> step-04 (resume review on - # the restored diff); from-scratch -> ready-for-dev -> step-03 - # (re-implement). Independent of the resolve agent having set it. - target_status = "in-review" if restore_patch else "ready-for-dev" - # Whether the writes below are the copy the re-driven session actually - # reads. Hoisted out of the record's condition because TWO decisions turn on - # it, and only one of them used to: the warning below, and the flip's - # REFUSAL one screen down, which was gated on `spec_path.is_file()` alone. - # Under isolation that readable file is the doomed worktree copy, so the - # refusal demanded a repair to the one file the re-drive destroys before - # reading anything — and demanded it even when `_redrive_spec_status` had - # already proven the committed spec carries the status the re-drive routes - # on. See `_spec_is_shared_with_the_redrive` for why an isolated unit's spec - # is nevertheless reachable when it sits in an artifact dir configured - # outside the project tree. - write_reaches_the_redrive = spec_reaches_the_redrive( - task, state, isolated_redrive=isolated_redrive - ) - # Narrowed to the case an operator can ACT on. Every isolated escalation - # carries a mounted `worktree_path` — `worktree_flow.escalate_unit` never - # clears it, and `keep_branch_and_escalate` deliberately leaves the worktree - # up — so gating on that alone fired this warning on 100% of re-arms under - # `isolation = "worktree"`: a per-configuration constant, not an event, and - # the same "trains the operator to scroll past the meaningful one" failure - # that the `flipped` read-back below and the `overwritten != old_baseline` - # guard were each narrowed to avoid. The remedy it prints ("commit the - # corrected spec") is already a no-op once the committed spec carries the - # target status, which is precisely when the re-drive reads what it needs. - # Suppression requires PROOF: an unreadable blob, a non-repo project, or any - # git fault leaves `""` and the record fires. The proof is read at - # `redrive_base_ref`, NOT at the code root's current `HEAD` — the two part - # company as soon as the operator checks out another branch while the - # escalation is paused, and this record now holds the resume. - # - # The branch rides along because the remedy needs it: on exactly the shape - # the ref fix rescues, "commit the corrected spec" without a branch sends - # the operator to commit again on the branch the re-drive does not read, and - # the next re-arm prints the same sentence. Empty for the migrated shape - # `redrive_base_ref` degrades to `HEAD` for, and the notice drops the - # clause rather than naming a ref it cannot source — and empty for an - # IN-PLACE re-drive, which has no branch to name at all. - # - # `redrive` is that second shape's discriminator, and it goes ON the record - # because the reader is out of process: `rearm_event_notice` renders from a - # journal line alone and cannot re-read the policy that produced it. One - # kind, two remedies. Isolated: the writes landed in a mount the re-drive - # discards, so the correction must be COMMITTED on the named branch. In - # place: the writes landed in the mount the escalated attempt recorded while - # the re-drive now reads the main checkout, so the correction must be made - # THERE — a commit is neither required nor sufficient. Telling the second - # operator to commit sends them to the wrong tree, which is the same class - # of silent loss this whole record exists to end. - # - # Spelled `target_branch` and NOT `base`, because `diagnostics` routes the - # scrub by field NAME: `target_branch` is already in `_JOURNAL_ALIAS_FIELDS` - # under the `branch` namespace (with no journal producer until now), while - # any new spelling falls through to `scrub_json`, which waves an - # identifier-shaped branch name through verbatim. In a normal run - # `ensure_target_branch` has already journalled the same string as `branch`, - # so the egress backstop would repair it and disclose a `backstop_repairs` - # routing gap; in a truncated journal missing that event nothing would catch - # it and the branch would ship in a shareable bundle. `target` — the - # spelling the merge kinds use — is NOT available: `board-advance-*` puts a - # sprint STATUS in that same field, and routing is by name, so aliasing it - # to `branch` would pseudonymize statuses as branches. - if ( - not write_reaches_the_redrive - and _redrive_spec_status(state, task, isolated_redrive=isolated_redrive) - != target_status - ): - journal.append( - "rearm-spec-write-unreachable", - story_key=key, - spec_file=str(spec_path), - status=target_status, - target_branch=state.target_branch if isolated_redrive else "", - redrive="isolated" if isolated_redrive else "in-place", - ) - # Captured immediately before the FIRST write, so an abort further down can - # put the spec back exactly as found. Unreadable degrades to `None`: the - # writes below answer such a path with `False` rather than an exception, so - # there would be nothing to undo either. - try: - spec_before = spec_path.read_bytes() - except OSError: - spec_before = None - try: - flipped = verify.set_frontmatter_status( - spec_path, target_status, confine_root=task_spec_root(task, state) + spec_path: Path | None = None + + # ONE transaction, from the first spec write to the commit point. `save_state` IS + # that commit point: until it returns the run still calls this story ESCALATED, so + # any fault escaping this window left a spec re-armed on disk against a task that is + # not — the "one edit nothing else records" each sequenced refusal was written to + # avoid, reached instead by a `journal.append` OSError from the residue pass, a + # non-Git fault from the commits probe, or `save_state` itself failing. Only two of + # the aborts in here ever undid their own writes; guarding the window replaces both + # of those per-arm undos with one rule that covers every fault source in it. + # + # `except BaseException: ...; raise` rather than a `finally` with a flag, because + # the rollback must run on the ERROR path only and a bare `raise` re-raises the + # ORIGINAL fault untouched — the narrowed `verify.GitError` taxonomies inside stay + # narrowed, and a non-git fault from either probe still escapes as itself. + # + # The guard opens one line above the FIRST write rather than at the `spec_before` + # capture, because the residue pass, the advance and `save_state` all have to be + # covered too and they live outside that block. + # + # STATE THE SCOPE PRECISELY, because one branch in here is the counterexample to the + # loose reading: what this transaction restores is the SPEC's BYTES, from the first + # spec write onward. It is not "the tree as the re-arm found it". The sentinel-clear + # branch sits INSIDE the guard and UNLINKS a file, and that deletion is deliberately + # NOT undone — `_clear_sentinel` preserves a copy under `{run_dir}/sentinels/` and a + # retried resolve re-clears it idempotently, so re-creating it here would fight a + # gesture that is already safe to repeat. `spec_before` is `None` on that leg, so + # `_rollback_rearm` records `unknown` rather than `unchanged` and the operator + # surfaces make no claim about the file. Recording it as `unchanged` is precisely + # the bug that reading would produce: a notice naming a file this re-arm DELETED as + # proof the tree is untouched. + try: + if task.spec_file: + spec_path = task_spec_path(task, state) + # Stories mode only: a fixed-slug pre-planning-halt sentinel + # (`-unresolved.md` / `-ambiguous.md`) is cleared by deletion, not a + # status flip. Clear it ONLY when the run recorded this task AS a sentinel at + # detection time (`task.sentinel_kind`, stamped by StoriesEngine's pick-time + # wedge / post-dev read-back) — never by re-deriving from the basename. That + # keeps a real story spec that merely happens to be named `-unresolved.md`, + # or a *non-sentinel* escalation whose spec matches the convention, on the + # status-flip path so it is kept, not deleted. Gate on the run source too (the + # convention exists only in stories mode) and defensively re-confirm the + # on-disk name still matches the recorded slug before deleting. + sentinel_kind = task.sentinel_kind if state.source == "stories" else "" + if sentinel_kind and _sentinel_condition(spec_path, key) == sentinel_kind: + # a sentinel is cleared by deletion, not a status flip; drop the stale + # spec_file so the re-dispatch starts from PENDING (clean re-plan). + _clear_sentinel(run_dir, journal, spec_path, key, sentinel_kind) + task.spec_file = None + task.sentinel_kind = "" # verdict discharged; the re-dispatch is clean + # Deleting the sentinel does not make the re-plan produce a different one: + # the correction that does lives UPSTREAM, in the `SPEC.md` / `stories.yaml` + # the resolve skill sends the agent to instead of this file. That correction + # faces the same reachability gap the spec arm below measures, and faced NO + # gate at all — this arm cleared `spec_file` and fell through, so + # `write_reaches_the_redrive` was never computed and the resume was never + # held for a sentinel. An isolated re-drive then mounts fresh from + # `redrive_base_ref`, re-plans from a committed tree that never saw the + # edit, mints the same sentinel again, and the escalation is spent. + # + # Narrowed by PROOF for the reason the spec record below is, and the need is + # sharper here: `stories_reach_the_redrive` answers "unreachable" for EVERY + # isolated stories run whose spec folder sits inside the project, which is + # every one we author. Gating on it alone would fire — and hold the resume — + # on 100% of isolated sentinel re-arms, a per-configuration constant rather + # than an event. `_redrive_reads_the_upstream_artifacts` is what makes it an + # event: it fires only while this checkout still holds upstream bytes the + # ref the re-drive mounts from does not. + # + # No `redrive` discriminator, unlike the spec record: this one has a single + # remedy because it has a single reachable shape. An in-place re-drive reads + # the main checkout's working tree, which is exactly where `cwd=project` put + # the correction, so `stories_reach_the_redrive` short-circuits that leg to + # reachable and no record is written for it at all. + if not stories_reach_the_redrive( + task, state, isolated_redrive=isolated_redrive + ) and not _redrive_reads_the_upstream_artifacts(state): + journal.append( + "rearm-upstream-write-unreachable", + story_key=key, + # `task_stories_root` names the tree the RUN owns; the correction + # lands in the checkout the resolve session ran in. Both are the + # project on this leg unless a mount is recorded, and the operator + # needs the folder to act, so the record carries the folder the + # remedy is about rather than the run's read locator. + stories_root=str(_upstream_artifacts_folder(state)), + target_branch=state.target_branch, + ) + else: + # A WORKTREE-LOCAL spec's writes below land in the unit's worktree + # (`task_spec_path`) — which the re-drive destroys before reading anything. + # A re-armed task (phase PENDING, `defer_reason` cleared, and no resumable + # session because `generation` was just bumped) falls to + # `engine._finish_inflight`'s final arm, which calls `discard_worktree` and + # lets `_run_story` mount a fresh one. The re-driven session then resolves + # its spec through `verify.resolve_spec_path(task.spec_file, + # workspace.paths)` (`engine._dispatched_spec_for_attempt`), and under + # isolation `workspace.paths` is rebased onto that FRESH worktree, which + # checks out TRACKED files only. So the re-drive reads the COMMITTED spec. + # + # No working-tree write reaches it — not this one, and not a write to the + # main checkout either: the fresh worktree comes from git rather than from a + # copy of that tree, and `seed_adapter_defaults` seeds adapter config files, + # not the output folder. The channel that DOES work is the human committing + # the corrected spec from the resolve session, which runs with `cwd=project`. + # The writes below are kept (they are correct for the in-place case, and + # harmless here), but the operator is told — a flip that cannot land is + # exactly the silent re-wedge #640(b) exists to end. + # + # "Worktree-local" is the load-bearing qualifier, and isolation does not + # imply it: an artifact dir configured OUTSIDE the project tree is shared + # across checkouts by `ProjectPaths.rebased`, so a spec that landed there is + # one file the fresh worktree reads through the very absolute path this + # writes to. `_spec_is_shared_with_the_redrive` carves out that case, and only + # that one: the main checkout's copy is outside the worktree too, and stays + # unreachable because the re-drive measures it against worktree-local roots. + # Route /bmad-build-auto via the spec's frontmatter status (decision + # table): patch-restore -> in-review -> step-04 (resume review on + # the restored diff); from-scratch -> ready-for-dev -> step-03 + # (re-implement). Independent of the resolve agent having set it. + target_status = "in-review" if restore_patch else "ready-for-dev" + # Whether the writes below are the copy the re-driven session actually + # reads. Hoisted out of the record's condition because TWO decisions turn on + # it, and only one of them used to: the warning below, and the flip's + # REFUSAL one screen down, which was gated on `spec_path.is_file()` alone. + # Under isolation that readable file is the doomed worktree copy, so the + # refusal demanded a repair to the one file the re-drive destroys before + # reading anything — and demanded it even when `_redrive_spec_status` had + # already proven the committed spec carries the status the re-drive routes + # on. See `_spec_is_shared_with_the_redrive` for why an isolated unit's spec + # is nevertheless reachable when it sits in an artifact dir configured + # outside the project tree. + write_reaches_the_redrive = spec_reaches_the_redrive( + task, state, isolated_redrive=isolated_redrive ) - # `set_frontmatter_status` answers "nothing to change" with `False` - # for FOUR causes, not three — its own docstring lists them: no file, - # no frontmatter block, no top-level `status:`, and ALREADY AT THE - # TARGET (`_edit_frontmatter_block` returns None on - # `original[key] == value`). Only the first three are failures. The - # fourth is an ordinary, fully-successful re-arm: a second resolve - # cycle on an already-flipped spec, or the documented - # `resolve --no-interactive` flow where a human fixed the spec - # themselves — the case the comment above calls "Independent of the - # resolve agent having set it". Journalling it fired the operator - # warning ("could not be re-opened … may re-wedge on it") on a spec - # that was byte-identical and CORRECT, which is the "trains the - # operator to scroll past the meaningful one" failure the re-stamp's - # `overwritten != old_baseline` guard exists to prevent one screen - # below. Read the status back to tell the two apart: `read_frontmatter` - # degrades a missing/unreadable/unparseable spec to `{}` and `status_of` - # then answers `""`, so all three real failures still record. - if not flipped and verify.status_of(verify.read_frontmatter(spec_path)) != ( - target_status + # Narrowed to the case an operator can ACT on. Every isolated escalation + # carries a mounted `worktree_path` — `worktree_flow.escalate_unit` never + # clears it, and `keep_branch_and_escalate` deliberately leaves the worktree + # up — so gating on that alone fired this warning on 100% of re-arms under + # `isolation = "worktree"`: a per-configuration constant, not an event, and + # the same "trains the operator to scroll past the meaningful one" failure + # that the `flipped` read-back below and the `overwritten != old_baseline` + # guard were each narrowed to avoid. The remedy it prints ("commit the + # corrected spec") is already a no-op once the committed spec carries the + # target status, which is precisely when the re-drive reads what it needs. + # Suppression requires PROOF: an unreadable blob, a non-repo project, or any + # git fault leaves `""` and the record fires. The proof is read at + # `redrive_base_ref`, NOT at the code root's current `HEAD` — the two part + # company as soon as the operator checks out another branch while the + # escalation is paused, and this record now holds the resume. + # + # The branch rides along because the remedy needs it: on exactly the shape + # the ref fix rescues, "commit the corrected spec" without a branch sends + # the operator to commit again on the branch the re-drive does not read, and + # the next re-arm prints the same sentence. Empty for the migrated shape + # `redrive_base_ref` degrades to `HEAD` for, and the notice drops the + # clause rather than naming a ref it cannot source — and empty for an + # IN-PLACE re-drive, which has no branch to name at all. + # + # `redrive` is that second shape's discriminator, and it goes ON the record + # because the reader is out of process: `rearm_event_notice` renders from a + # journal line alone and cannot re-read the policy that produced it. One + # kind, two remedies. Isolated: the writes landed in a mount the re-drive + # discards, so the correction must be COMMITTED on the named branch. In + # place: the writes landed in the mount the escalated attempt recorded while + # the re-drive now reads the main checkout, so the correction must be made + # THERE — a commit is neither required nor sufficient. Telling the second + # operator to commit sends them to the wrong tree, which is the same class + # of silent loss this whole record exists to end. + # + # Spelled `target_branch` and NOT `base`, because `diagnostics` routes the + # scrub by field NAME: `target_branch` is already in `_JOURNAL_ALIAS_FIELDS` + # under the `branch` namespace (with no journal producer until now), while + # any new spelling falls through to `scrub_json`, which waves an + # identifier-shaped branch name through verbatim. In a normal run + # `ensure_target_branch` has already journalled the same string as `branch`, + # so the egress backstop would repair it and disclose a `backstop_repairs` + # routing gap; in a truncated journal missing that event nothing would catch + # it and the branch would ship in a shareable bundle. `target` — the + # spelling the merge kinds use — is NOT available: `board-advance-*` puts a + # sprint STATUS in that same field, and routing is by name, so aliasing it + # to `branch` would pseudonymize statuses as branches. + if ( + not write_reaches_the_redrive + and _redrive_spec_status(state, task, isolated_redrive=isolated_redrive) + != target_status ): - # Discarding that return is how the flip - # became a SILENT no-op: the re-drive is dispatched anyway, step-01 - # reads the unchanged terminal status, routes the session to "ingest - # as context, do not resume", and the story re-wedges with nothing on - # the record. The `FrontmatterWriteError` arm below covers only the - # shapes that RAISE; this covers the ones that lie quietly. - # `refused` is written ON the record because ONE kind now covers - # two outcomes and the operator surfaces must tell them apart — - # they read the journal OUT OF PROCESS, with neither the task nor - # the tree to re-derive it from. Printing the refusal's remedy - # ("add a top-level `status:`") for a re-arm that COMPLETED sends - # the human to repair a file nothing will read. - refused = spec_path.is_file() and write_reaches_the_redrive journal.append( - "rearm-spec-flip-skipped", + "rearm-spec-write-unreachable", story_key=key, spec_file=str(spec_path), status=target_status, - refused=refused, + target_branch=state.target_branch if isolated_redrive else "", + redrive="isolated" if isolated_redrive else "in-place", ) - # ...and then ABORT — but only for a spec that IS a readable file - # here AND is the copy the re-drive reads. The first half is the same - # `is_file` split the baseline re-stamp below already draws, and for - # the same reason. On THAT shape the failure is - # a REPAIR that did not land on the very file the re-drive reads, so it - # aborts for the same reason the `FrontmatterWriteError` arm does: - # journalling alone left the two default surfaces telling the operator - # "re-armed " and resuming in the same gesture, so the record's - # own imperative was already unactionable when it rendered — while - # step-01's contract for what reaches here is not a maybe. A spec with - # no `status:` HALTs blocked on `unrecognized status in existing story - # file`; one still carrying the escalated attempt's terminal status - # routes to "ingest as context, do not resume". Either way the re-drive - # re-wedges and the escalation is burned. Refusing keeps it armed: nothing - # is persisted yet (`save_state` runs below), the spec is byte-identical - # (the `## Auto Run Result` strip is deliberately sequenced AFTER this - # check so an abort leaves nothing half-done), and the human fixes the - # frontmatter and re-runs resolve. - # - # A spec that is NOT a file from here keeps warn-and-continue, because - # there the flip's failure says nothing about what the re-drive will - # read: `spec_file` is persisted RELATIVE to a worktree, an isolated - # task's worktree may already be gone, and the re-drive mounts a fresh - # one and reads the COMMITTED spec regardless. Aborting on it would - # refuse the re-arms that the `rearm-baseline-restamp-skipped` and - # `rearm-spec-write-unreachable` records exist to report rather than - # prevent — an unreadable path is an observation, and observations - # degrade. - # - # A worktree-local spec that IS readable takes that same lane, for a - # sharper version of the same reason: `task_spec_root` anchors this - # write on the mounted worktree, so the readable file is the copy the - # re-drive DISCARDS. The refusal's own remedy could not fix anything - # there — an operator who added a `status:` to that file and re-ran - # resolve would flip a spec that is deleted before it is read, while - # the committed spec, the one thing that decides routing, went - # untouched. Worse, the refusal fired even when the correction was - # already committed: `_redrive_spec_status` had just PROVEN the - # re-drive routes correctly, and the re-arm was refused anyway over an - # obsolete copy. The real remedy on that shape is - # `rearm-spec-write-unreachable`'s ("commit the corrected spec"), - # which fires from the block above on exactly the legs that need it - # and now holds the resume rather than merely printing. - # - # The record is written on BOTH sides of that split: the abort message - # reaches stderr only, and the journal is the run's audit trail — - # `_echo_rearm_events` surfaces it from a `finally` on this path. - if refused: - raise RearmError( - f"cannot re-open story spec {spec_path} to `{target_status}` " - "for the re-drive: it has no frontmatter `status:` this re-arm " - "can set, so the re-driven session would wedge on the status " - "it reads — add a top-level `status:` to the spec's " - "frontmatter block, then re-run resolve" - ) - # drop the stale `## Auto Run Result` section along with the status flip - # (mirrors engine._reset_spec_for_repair): find_result_artifact keys on - # that heading, so leaving it would let the re-driven session's first - # save of the spec parse as the prior attempt's terminal outcome. + # Captured immediately before the FIRST write, so an abort further down can + # put the spec back exactly as found. # - # Sequenced AFTER the read-back check above, not with the flip it mirrors: - # that check now raises, and an aborted re-arm must leave the spec exactly - # as it found it — a stripped result section on a spec the re-arm then - # refused would be the one edit nothing else records. - devcontract.strip_auto_run_result( - spec_path, confine_root=task_spec_root(task, state) - ) - except verify.FrontmatterWriteError as e: - # The spec reads fine but carries `status:` in a shape no line - # edit can move (a block scalar, a flow mapping, a value continued - # on the next line). This used to be a silent no-op on a bool - # nobody read: the re-drive was dispatched anyway, step-01 saw the - # unchanged terminal status and routed the session to "ingest as - # context, do not resume", and the story re-wedged with nothing on - # the record explaining why. Abort here for the same reason as - # below, with the remedy this cause actually has. - raise RearmError( - f"cannot re-open story spec {spec_path} for the re-drive: {e} " - f"— the re-drive would repeat the wedge it is meant to clear" - ) from e - except (OSError, UnicodeDecodeError) as e: - # Both helpers re-read the spec as UTF-8; an undecodable PRESENT - # spec is a first-class escalation state (resolve_story_spec - # degrades it to a wedge), so it can reach this flip. Without the - # flip the re-drive would just re-wedge — abort BEFORE any state - # is persisted (save_state runs below) with an actionable error - # instead of a traceback; the escalation stays armed for a retry. + # A path that is NOT a file here degrades to `None`, and that degrade is + # sound for the reason it always was: every writer below answers such a + # path with `False` rather than an exception, so there is nothing to undo. + # A missing spec, a dangling link and a directory all land there. # - # ...and this arm is the SECOND refusal that can fire after a write has - # landed, which the sequencing argument above does not cover. It guards - # BOTH helpers, and `strip_auto_run_result` is the later one: by the - # time its own read/decode or its atomic write faults (an - # `atomic_write_bytes_confined` that cannot land — ENOSPC, EIO, a - # component swapped for a link under the `O_NOFOLLOW` walk — or a spec - # replaced under us between the two writes), the flip has already been - # published and `save_state` has not. Ordering the strip after the - # read-back check bought that check its byte-identical abort; it buys - # this one nothing, because the fault is IN the strip. So the same undo - # the re-stamp carries applies here, on the same terms. + # A path that IS a file whose bytes could not be read is the opposite + # case, and it must FAIL BEFORE WRITING. The read below is one syscall + # among many against a file three later writers open independently, so a + # transient fault (EIO on a network mount, a momentary EACCES, ENFILE + # under load) can be followed by writes that all succeed — and the abort + # that follows would then find `spec_before is None`, record `unknown`, + # and re-raise with the flip PUBLISHED and nothing put back. That is + # DW-79/DW-83 reached through the transaction's own preimage. Refusing + # keeps the escalation armed for a retry. # - # On the arm's other shape — the flip itself faulting on an - # unreadable/undecodable spec — nothing was written, `spec_before` still - # equals the bytes on disk, and `_restore_rearmed_spec` proves that and - # returns without touching the file or its mtime. - _restore_rearmed_spec(spec_path, spec_before, task, state) - raise RearmError( - f"cannot re-open story spec {spec_path} for the re-drive " - f"({e.__class__.__name__}: {e}) — fix or replace the file " - f"(it must be readable UTF-8), then re-run resolve" - ) from e - - # A previous restore latch is being replaced (or re-latched onto the same - # patch): the abandoned attempt applied that patch, so its NEW files sit - # untracked in the tree right now. The refresh below would capture them as - # "pre-existing" — after which every rollback preserves them and - # finalize_commit's `add -A` sweeps the abandoned attempt into the corrected - # story's commit. Subtract them instead (issue #90). - # - # Runs after the spec block for the same reason the refresh does (a cleared - # sentinel must not be snapshotted), and before it because it feeds it. - # Nothing is deleted here: the re-drive's reset (verify.safe_rollback) removes - # whatever the refreshed snapshot no longer blesses, at the right moment. - # The CODE tree, not `state.project`: every git read below (and every baseline - # the proof-of-work gate later measures against) must name the repository the - # dev writer stamps. - # - # That is `paths.repo_root` for every run this function can be reached from, but - # NOT because `paths.repo_root == workspace.root` universally — it does not. - # `Workspace.default` sets `root=paths.repo_root`, while the isolation constructor - # mounts `root=/worktrees/` and rebases a fresh `ProjectPaths` onto - # it, so under `isolation = "worktree"` the run-level `repo_root` is the main - # checkout and the baseline is stamped in the worktree. - # - # `bmadconfig.worktree_isolation_conflict` refuses worktree isolation beside a - # `repo_root:` OVERRIDE — a narrower fact than it looks. It forces - # `repo_root == project`; it says nothing about `repo_root` vs `workspace.root`. - # Under plain isolation with NO override those two still diverge and isolation is - # ON, so "wherever the roots could diverge, isolation is off" is false, and a rule - # built on it licenses treating `state.code_root` as the tree the dev writer - # stamped — which under isolation it is not. - # - # What is true, and the only claim to carry forward: `repo_root == project` in - # every reachable configuration, so reading HEAD here is right for the in-place - # case; and under isolation this value is deliberately SUPERSEDED rather than - # relied on — `engine._finish_inflight` discards the worktree and `_dev_phase` - # re-stamps `task.baseline_commit` from the fresh worktree's HEAD before any gate - # reads it. Do not carry an identity into new code; carry this argument. - # - # A pre-upgrade state.json with no recorded root degrades to `project` exactly as - # before. - repo = state.code_root - stale_residue = _stale_restore_residue(repo, journal, key, old_latch, old_baseline) - - # Advance the attempt baseline to the CODE TREE's current HEAD (`repo`, above) - # and refresh the untracked snapshot: whatever the human-driven resolve session left on the - # branch (a committed fixture, a corrected ledger, ...) is authorized input - # for the re-drive, not failed-attempt debris. Without this, the re-drive's - # reset-to-baseline in engine._rollback_or_pause parks the resolution - # commits on an attempt-preserve ref and rebuilds against a tree that - # contradicts the corrected spec — the re-driven dev session then hits the - # very gap the human just resolved. Best-effort: on a git failure the old - # baseline stands (the redrive rollback path tolerates a stale baseline; it - # just loses this protection). - # Runs AFTER the spec block so a just-cleared stories sentinel (an untracked - # file removed above) is not captured into baseline_untracked as a phantom - # pre-existing untracked file. The two locals are computed before either task - # field is assigned, so a failure on either git call can't advance - # baseline_commit while baseline_untracked stays stale, or vice versa. - advanced = False - try: - head = verify.rev_parse_head(repo) - untracked = sorted(verify.untracked_files(repo) - stale_residue) - except verify.GitError as e: - # `verify.GitError` is a TOTAL replacement for the `except Exception` that - # stood here, not a narrowing that leaks: both calls go through `_run_git`, - # which translates spawn (`GitSpawnError`), timeout (`GitTimeoutError`) and - # decode faults into this one taxonomy, and a non-zero rc into a plain - # `GitError`. Still swallowed rather than raised — a project that is not a - # git repo must not fail re-arm — but no longer SILENT: the degrade is the - # difference between "the re-drive starts from the resolution" and "it - # rebuilds against the tree the human just corrected away", and the - # re-stamp below now refuses to paper over it. - journal.append( - "rearm-baseline-advance-failed", - story_key=key, - repo=str(repo), - baseline=old_baseline or "", - error=f"{e.__class__.__name__}: {e}", - ) - else: - task.baseline_commit = head - task.baseline_untracked = untracked - advanced = True - - # Re-stamp the spec's own baseline to the advanced one, on BOTH re-drive legs. - # - # The patch-restore leg needs it because the in-review route skips step-03 — - # the only step that stamps `baseline_revision` — so without it the re-driven - # step-04 would build its review diff (and, on an intent-gap/bad-spec - # re-triage, revert) "since" the ORIGINAL pre-attempt sha, clawing back the - # very resolve-session commits the advance above just blessed as the re-drive's - # starting point. - # - # The from-scratch leg gets it too (#640a). Its step-03 re-stamps the key - # itself, so the write is redundant on the happy path — but only ON that path: - # until step-03 runs, the spec carries the escalated attempt's sha, and every - # gate that reads a claimed baseline before then reads a stale one. The cost is - # recorded rather than hidden: re-stamping removes the gate's INDEPENDENT - # signal on this leg (it then compares a value the orchestrator itself wrote), - # so a claim that genuinely diverged is journalled on the way out instead of - # being silently normalized. - # - # Gated on `advanced`, not on truthiness of `task.baseline_commit`: a failed - # advance leaves the OLD sha in that field, which passes a truthiness test - # identically to a freshly advanced one. Writing it would make spec and task - # agree on a stale value — the one state in which nothing downstream can tell - # that the advance never happened, and the re-drive rebuilds from the wrong - # point with no error anywhere. Skipping keeps the failure legible (the degrade - # is journalled above) and keeps re-arm non-fatal outside a repo. - # - # Loud on WRITE failure: a silently stale spec baseline is exactly the hazard - # being closed. - # - # Guarded on `is_file` FIRST, because a spec this process cannot reach is not a - # write failure here — it is a SILENT one. Both frontmatter writers answer such a - # path with `False` rather than an exception (`verify.set_frontmatter_status`, - # `verify.set_frontmatter_field`), so without a check the re-stamp no-ops with - # nothing on the record and the spec keeps the escalated attempt's sha. - # - # `task_spec_path` re-anchors the recorded path before we get here, which is what - # makes `is_file` mean what it says. Resolved raw it meant something else and worse: - # `spec_file` is persisted RELATIVE to the worktree for an isolated task, and the - # main checkout carries the same layout, so the check passed on the wrong file and - # the write landed there. The restore leg cannot reach any of this (its precondition - # rejects a truthy `task.worktree_path`); the from-scratch leg has no such guard, - # which is exactly why that precondition has to exist. - # - # `is_file` is necessary but not sufficient: a spec that EXISTS with no frontmatter - # block also returns `False` from both writers. That shape is caught by the flip's - # `flipped` check above and, here, by `overwritten` staying empty. - if task.spec_file: - spec_path = task_spec_path(task, state) - if not spec_path.is_file(): - # OUTSIDE the `advanced` gate on purpose. Nesting this record inside it - # made the two #640 legs shadow each other: on a project that is not a - # repo the advance fails, `advanced` is False, and an unreadable spec - # then produced NO record at all — the journal blamed git while the - # status flip above had silently no-opped for an entirely different - # reason. The two degrades compose; they do not substitute. + # Gated on the SAME PAIR as the flip's refusal one screen below + # (`spec_path.is_file() and write_reaches_the_redrive`), because it is the + # same abort-vs-warn decision about the same file and the two must not + # disagree. `is_file` alone is not enough: under isolation the readable + # file is the worktree copy the re-drive DESTROYS before reading anything, + # so an abort there demands a repair to a file nothing opens — its remedy + # cannot change what the re-drive reads, and it costs the operator the + # interactive resolve session over a spec whose real reachability record + # (`rearm-spec-write-unreachable`) has already been written above. On that + # shape the unreadable preimage is an OBSERVATION, and observations + # degrade: `spec_before` stays `None`, the writes below no-op or land on a + # doomed copy, and any later abort records `unknown` rather than claiming + # a file it never captured. + try: + spec_before = spec_path.read_bytes() + except OSError as e: + if spec_path.is_file() and write_reaches_the_redrive: + raise RearmError( + f"cannot read story spec {spec_path} before re-opening it for " + f"the re-drive ({e.__class__.__name__}: {e}) — the re-arm " + "refuses to write a spec it could not capture first, since a " + "later abort would have nothing to put back; fix or replace " + "the file, then re-run resolve" + ) from e + spec_before = None + try: + flipped = verify.set_frontmatter_status( + spec_path, target_status, confine_root=task_spec_root(task, state) + ) + # `set_frontmatter_status` answers "nothing to change" with `False` + # for FOUR causes, not three — its own docstring lists them: no file, + # no frontmatter block, no top-level `status:`, and ALREADY AT THE + # TARGET (`_edit_frontmatter_block` returns None on + # `original[key] == value`). Only the first three are failures. The + # fourth is an ordinary, fully-successful re-arm: a second resolve + # cycle on an already-flipped spec, or the documented + # `resolve --no-interactive` flow where a human fixed the spec + # themselves — the case the comment above calls "Independent of the + # resolve agent having set it". Journalling it fired the operator + # warning ("could not be re-opened … may re-wedge on it") on a spec + # that was byte-identical and CORRECT, which is the "trains the + # operator to scroll past the meaningful one" failure the re-stamp's + # `overwritten != old_baseline` guard exists to prevent one screen + # below. Read the status back to tell the two apart: `read_frontmatter` + # degrades a missing/unreadable/unparseable spec to `{}` and `status_of` + # then answers `""`, so all three real failures still record. + if not flipped and verify.status_of(verify.read_frontmatter(spec_path)) != ( + target_status + ): + # Discarding that return is how the flip + # became a SILENT no-op: the re-drive is dispatched anyway, step-01 + # reads the unchanged terminal status, routes the session to "ingest + # as context, do not resume", and the story re-wedges with nothing on + # the record. The `FrontmatterWriteError` arm below covers only the + # shapes that RAISE; this covers the ones that lie quietly. + # `refused` is written ON the record because ONE kind now covers + # two outcomes and the operator surfaces must tell them apart — + # they read the journal OUT OF PROCESS, with neither the task nor + # the tree to re-derive it from. Printing the refusal's remedy + # ("add a top-level `status:`") for a re-arm that COMPLETED sends + # the human to repair a file nothing will read. + refused = spec_path.is_file() and write_reaches_the_redrive + journal.append( + "rearm-spec-flip-skipped", + story_key=key, + spec_file=str(spec_path), + status=target_status, + refused=refused, + ) + # ...and then ABORT — but only for a spec that IS a readable file + # here AND is the copy the re-drive reads. The first half is the same + # `is_file` split the baseline re-stamp below already draws, and for + # the same reason. On THAT shape the failure is + # a REPAIR that did not land on the very file the re-drive reads, so it + # aborts for the same reason the `FrontmatterWriteError` arm does: + # journalling alone left the two default surfaces telling the operator + # "re-armed " and resuming in the same gesture, so the record's + # own imperative was already unactionable when it rendered — while + # step-01's contract for what reaches here is not a maybe. A spec with + # no `status:` HALTs blocked on `unrecognized status in existing story + # file`; one still carrying the escalated attempt's terminal status + # routes to "ingest as context, do not resume". Either way the re-drive + # re-wedges and the escalation is burned. Refusing keeps it armed: nothing + # is persisted yet (`save_state` runs below), the spec is byte-identical + # (the `## Auto Run Result` strip is deliberately sequenced AFTER this + # check so an abort leaves nothing half-done), and the human fixes the + # frontmatter and re-runs resolve. + # + # A spec that is NOT a file from here keeps warn-and-continue, because + # there the flip's failure says nothing about what the re-drive will + # read: `spec_file` is persisted RELATIVE to a worktree, an isolated + # task's worktree may already be gone, and the re-drive mounts a fresh + # one and reads the COMMITTED spec regardless. Aborting on it would + # refuse the re-arms that the `rearm-baseline-restamp-skipped` and + # `rearm-spec-write-unreachable` records exist to report rather than + # prevent — an unreadable path is an observation, and observations + # degrade. + # + # A worktree-local spec that IS readable takes that same lane, for a + # sharper version of the same reason: `task_spec_root` anchors this + # write on the mounted worktree, so the readable file is the copy the + # re-drive DISCARDS. The refusal's own remedy could not fix anything + # there — an operator who added a `status:` to that file and re-ran + # resolve would flip a spec that is deleted before it is read, while + # the committed spec, the one thing that decides routing, went + # untouched. Worse, the refusal fired even when the correction was + # already committed: `_redrive_spec_status` had just PROVEN the + # re-drive routes correctly, and the re-arm was refused anyway over an + # obsolete copy. The real remedy on that shape is + # `rearm-spec-write-unreachable`'s ("commit the corrected spec"), + # which fires from the block above on exactly the legs that need it + # and now holds the resume rather than merely printing. + # + # The record is written on BOTH sides of that split: the abort message + # reaches stderr only, and the journal is the run's audit trail — + # `_echo_rearm_events` surfaces it from a `finally` on this path. + if refused: + raise RearmError( + f"cannot re-open story spec {spec_path} to `{target_status}` " + "for the re-drive: it has no frontmatter `status:` this re-arm " + "can set, so the re-driven session would wedge on the status " + "it reads — add a top-level `status:` to the spec's " + "frontmatter block, then re-run resolve" + ) + # drop the stale `## Auto Run Result` section along with the status flip + # (mirrors engine._reset_spec_for_repair): find_result_artifact keys on + # that heading, so leaving it would let the re-driven session's first + # save of the spec parse as the prior attempt's terminal outcome. + # + # Sequenced AFTER the read-back check above, not with the flip it mirrors: + # that check now raises, and an aborted re-arm must leave the spec exactly + # as it found it — a stripped result section on a spec the re-arm then + # refused would be the one edit nothing else records. + devcontract.strip_auto_run_result( + spec_path, confine_root=task_spec_root(task, state) + ) + except verify.FrontmatterWriteError as e: + # The spec reads fine but carries `status:` in a shape no line + # edit can move (a block scalar, a flow mapping, a value continued + # on the next line). This used to be a silent no-op on a bool + # nobody read: the re-drive was dispatched anyway, step-01 saw the + # unchanged terminal status and routed the session to "ingest as + # context, do not resume", and the story re-wedged with nothing on + # the record explaining why. Abort here for the same reason as + # below, with the remedy this cause actually has. + raise RearmError( + f"cannot re-open story spec {spec_path} for the re-drive: {e} " + f"— the re-drive would repeat the wedge it is meant to clear" + ) from e + except (OSError, UnicodeDecodeError) as e: + # Both helpers re-read the spec as UTF-8; an undecodable PRESENT + # spec is a first-class escalation state (resolve_story_spec + # degrades it to a wedge), so it can reach this flip. Without the + # flip the re-drive would just re-wedge — abort BEFORE any state + # is persisted (save_state runs below) with an actionable error + # instead of a traceback; the escalation stays armed for a retry. + # + # Two shapes reach this arm and the transaction guard below covers both. + # On the flip's own read/decode fault nothing was written, so the rollback + # proves that and leaves the file and its mtime alone. A fault raised + # inside `strip_auto_run_result` is raised with the flip already PUBLISHED + # — an `atomic_write_bytes_confined` that cannot land (ENOSPC, EIO, a + # component swapped for a link under the `O_NOFOLLOW` walk), or a spec + # replaced under us between the two writes. Ordering the strip after the + # read-back check bought that check its byte-identical abort; it buys this + # one nothing, because the fault is IN the strip. + raise RearmError( + f"cannot re-open story spec {spec_path} for the re-drive " + f"({e.__class__.__name__}: {e}) — fix or replace the file " + f"(it must be readable UTF-8), then re-run resolve" + ) from e + + # A previous restore latch is being replaced (or re-latched onto the same + # patch): the abandoned attempt applied that patch, so its NEW files sit + # untracked in the tree right now. The refresh below would capture them as + # "pre-existing" — after which every rollback preserves them and + # finalize_commit's `add -A` sweeps the abandoned attempt into the corrected + # story's commit. Subtract them instead (issue #90). + # + # Runs after the spec block for the same reason the refresh does (a cleared + # sentinel must not be snapshotted), and before it because it feeds it. + # Nothing is deleted here: the re-drive's reset (verify.safe_rollback) removes + # whatever the refreshed snapshot no longer blesses, at the right moment. + # The CODE tree, not `state.project`: every git read below (and every baseline + # the proof-of-work gate later measures against) must name the repository the + # dev writer stamps. + # + # That is `paths.repo_root` for every run this function can be reached from, but + # NOT because `paths.repo_root == workspace.root` universally — it does not. + # `Workspace.default` sets `root=paths.repo_root`, while the isolation constructor + # mounts `root=/worktrees/` and rebases a fresh `ProjectPaths` onto + # it, so under `isolation = "worktree"` the run-level `repo_root` is the main + # checkout and the baseline is stamped in the worktree. + # + # `bmadconfig.worktree_isolation_conflict` refuses worktree isolation beside a + # `repo_root:` OVERRIDE — a narrower fact than it looks. It forces + # `repo_root == project`; it says nothing about `repo_root` vs `workspace.root`. + # Under plain isolation with NO override those two still diverge and isolation is + # ON, so "wherever the roots could diverge, isolation is off" is false, and a rule + # built on it licenses treating `state.code_root` as the tree the dev writer + # stamped — which under isolation it is not. + # + # What is true, and the only claim to carry forward: `repo_root == project` in + # every reachable configuration, so reading HEAD here is right for the in-place + # case; and under isolation this value is deliberately SUPERSEDED rather than + # relied on — `engine._finish_inflight` discards the worktree and `_dev_phase` + # re-stamps `task.baseline_commit` from the fresh worktree's HEAD before any gate + # reads it. Do not carry an identity into new code; carry this argument. + # + # A pre-upgrade state.json with no recorded root degrades to `project` exactly as + # before. + repo = state.code_root + stale_residue = _stale_restore_residue(repo, journal, key, old_latch, old_baseline) + + # Advance the attempt baseline to the CODE TREE's current HEAD (`repo`, above) + # and refresh the untracked snapshot: whatever the human-driven resolve session left on the + # branch (a committed fixture, a corrected ledger, ...) is authorized input + # for the re-drive, not failed-attempt debris. Without this, the re-drive's + # reset-to-baseline in engine._rollback_or_pause parks the resolution + # commits on an attempt-preserve ref and rebuilds against a tree that + # contradicts the corrected spec — the re-driven dev session then hits the + # very gap the human just resolved. Best-effort: on a git failure the old + # baseline stands (the redrive rollback path tolerates a stale baseline; it + # just loses this protection). + # Runs AFTER the spec block so a just-cleared stories sentinel (an untracked + # file removed above) is not captured into baseline_untracked as a phantom + # pre-existing untracked file. The two locals are computed before either task + # field is assigned, so a failure on either git call can't advance + # baseline_commit while baseline_untracked stays stale, or vice versa. + advanced = False + try: + head = verify.rev_parse_head(repo) + untracked = sorted(verify.untracked_files(repo) - stale_residue) + except verify.GitError as e: + # `verify.GitError` is a TOTAL replacement for the `except Exception` that + # stood here, not a narrowing that leaks: both calls go through `_run_git`, + # which translates spawn (`GitSpawnError`), timeout (`GitTimeoutError`) and + # decode faults into this one taxonomy, and a non-zero rc into a plain + # `GitError`. Still swallowed rather than raised — a project that is not a + # git repo must not fail re-arm — but no longer SILENT: the degrade is the + # difference between "the re-drive starts from the resolution" and "it + # rebuilds against the tree the human just corrected away", and the + # re-stamp below now refuses to paper over it. journal.append( - "rearm-baseline-restamp-skipped", + "rearm-baseline-advance-failed", story_key=key, - spec_file=str(spec_path), - baseline=task.baseline_commit or "", + repo=str(repo), + baseline=old_baseline or "", + error=f"{e.__class__.__name__}: {e}", ) - elif advanced and task.baseline_commit: - try: - # Read through the same reader both consumers of a claimed baseline use, - # so what gets journalled as "overwritten" is the value the gate would - # have judged — not whichever key happened to be inspected here (#716). - # - # INSIDE the try, with the write it describes. `read_frontmatter` opens - # the file itself, so an OSError here would otherwise escape as a - # traceback from the one block whose whole contract is to turn a spec - # this re-arm cannot move into an actionable `RearmError`. What it does - # NOT rescue: `read_frontmatter` DEGRADES an unparseable YAML block to - # `{}` rather than raising, so on such a spec `overwritten` is `""`, the - # guard below is falsy, and no divergence record is written even though - # the insert lands. That is the reader's deliberate observe-degrade - # contract, not something to defeat here — the value is unknowable, and - # inventing one would be worse than the silence. - overwritten = auto_dev_baseline_of(verify.read_frontmatter(spec_path)) - verify.set_frontmatter_field( - spec_path, - "baseline_revision", - task.baseline_commit, - confine_root=task_spec_root(task, state), - ) - except (OSError, UnicodeDecodeError, verify.FrontmatterWriteError) as e: - # FrontmatterWriteError joins the tuple rather than getting its own - # arm: the remedy is the same sentence ("fix the file"), and the - # exception already says which shape it could not move. What matters - # is that it aborts here — the stale-baseline hazard this block exists - # to close is exactly what a swallowed write would leave behind. - # - # ...and that the abort leaves the spec as this re-arm FOUND it. This is - # the LAST of the two refusals that can fire after a write has landed — - # the flip and the result strip are both behind us, `save_state` is not — - # so it carries the undo the sequenced refusals get for free (the other - # is the spec block's `(OSError, UnicodeDecodeError)` arm, which the - # strip raises through after the flip has published). Without - # it a spec with a movable `status:` beside an unmovable - # `baseline_revision:` came back flipped to the re-drive's status and - # stripped of the terminal result, while the run still called the story - # escalated. - _restore_rearmed_spec(spec_path, spec_before, task, state) - raise RearmError( - f"cannot re-stamp baseline_revision on {spec_path} " - f"({e.__class__.__name__}: {e}) — fix the file, then re-run resolve" - ) from e - if overwritten and overwritten != old_baseline: - # Compared against `old_baseline` — what the RUN recorded for the - # escalated attempt — NOT against `task.baseline_commit`, which the - # advance above has already moved to the new HEAD. Measuring against the - # advanced value made this fire on every ordinary from-scratch re-arm - # whose resolve session committed anything: the spec and the run agreed - # exactly, and the operator was still told they diverged. A record that - # fires on the routine case is the "trains the operator to scroll past - # the meaningful one" failure the `restore` split exists to prevent. - # - # What survives is the real signal, on BOTH legs: the spec claimed a - # baseline the run never recorded. That is the only trace left of a - # divergence the gate can no longer report, because the re-stamp is - # about to normalize it away. + else: + task.baseline_commit = head + task.baseline_untracked = untracked + advanced = True + + # Re-stamp the spec's own baseline to the advanced one, on BOTH re-drive legs. + # + # The patch-restore leg needs it because the in-review route skips step-03 — + # the only step that stamps `baseline_revision` — so without it the re-driven + # step-04 would build its review diff (and, on an intent-gap/bad-spec + # re-triage, revert) "since" the ORIGINAL pre-attempt sha, clawing back the + # very resolve-session commits the advance above just blessed as the re-drive's + # starting point. + # + # The from-scratch leg gets it too (#640a). Its step-03 re-stamps the key + # itself, so the write is redundant on the happy path — but only ON that path: + # until step-03 runs, the spec carries the escalated attempt's sha, and every + # gate that reads a claimed baseline before then reads a stale one. The cost is + # recorded rather than hidden: re-stamping removes the gate's INDEPENDENT + # signal on this leg (it then compares a value the orchestrator itself wrote), + # so a claim that genuinely diverged is journalled on the way out instead of + # being silently normalized. + # + # Gated on `advanced`, not on truthiness of `task.baseline_commit`: a failed + # advance leaves the OLD sha in that field, which passes a truthiness test + # identically to a freshly advanced one. Writing it would make spec and task + # agree on a stale value — the one state in which nothing downstream can tell + # that the advance never happened, and the re-drive rebuilds from the wrong + # point with no error anywhere. Skipping keeps the failure legible (the degrade + # is journalled above) and keeps re-arm non-fatal outside a repo. + # + # Loud on WRITE failure: a silently stale spec baseline is exactly the hazard + # being closed. + # + # Guarded on `is_file` FIRST, because a spec this process cannot reach is not a + # write failure here — it is a SILENT one. Both frontmatter writers answer such a + # path with `False` rather than an exception (`verify.set_frontmatter_status`, + # `verify.set_frontmatter_field`), so without a check the re-stamp no-ops with + # nothing on the record and the spec keeps the escalated attempt's sha. + # + # `task_spec_path` re-anchors the recorded path before we get here, which is what + # makes `is_file` mean what it says. Resolved raw it meant something else and worse: + # `spec_file` is persisted RELATIVE to the worktree for an isolated task, and the + # main checkout carries the same layout, so the check passed on the wrong file and + # the write landed there. The restore leg cannot reach any of this (its precondition + # rejects a truthy `task.worktree_path`); the from-scratch leg has no such guard, + # which is exactly why that precondition has to exist. + # + # `is_file` is necessary but not sufficient: a spec that EXISTS with no frontmatter + # block also returns `False` from both writers. That shape is caught by the flip's + # `flipped` check above and, here, by `overwritten` staying empty. + if task.spec_file: + spec_path = task_spec_path(task, state) + if not spec_path.is_file(): + # OUTSIDE the `advanced` gate on purpose. Nesting this record inside it + # made the two #640 legs shadow each other: on a project that is not a + # repo the advance fails, `advanced` is False, and an unreadable spec + # then produced NO record at all — the journal blamed git while the + # status flip above had silently no-opped for an entirely different + # reason. The two degrades compose; they do not substitute. journal.append( - "rearm-baseline-restamped", + "rearm-baseline-restamp-skipped", story_key=key, spec_file=str(spec_path), - overwritten=overwritten, - baseline=task.baseline_commit, - restore=bool(restore_patch), + baseline=task.baseline_commit or "", ) + elif advanced and task.baseline_commit: + try: + # Read through the same reader both consumers of a claimed baseline use, + # so what gets journalled as "overwritten" is the value the gate would + # have judged — not whichever key happened to be inspected here (#716). + # + # INSIDE the try, with the write it describes. `read_frontmatter` opens + # the file itself, so an OSError here would otherwise escape as a + # traceback from the one block whose whole contract is to turn a spec + # this re-arm cannot move into an actionable `RearmError`. What it does + # NOT rescue: `read_frontmatter` DEGRADES an unparseable YAML block to + # `{}` rather than raising, so on such a spec `overwritten` is `""`, the + # guard below is falsy, and no divergence record is written even though + # the insert lands. That is the reader's deliberate observe-degrade + # contract, not something to defeat here — the value is unknowable, and + # inventing one would be worse than the silence. + overwritten = auto_dev_baseline_of(verify.read_frontmatter(spec_path)) + verify.set_frontmatter_field( + spec_path, + "baseline_revision", + task.baseline_commit, + confine_root=task_spec_root(task, state), + ) + except (OSError, UnicodeDecodeError, verify.FrontmatterWriteError) as e: + # FrontmatterWriteError joins the tuple rather than getting its own + # arm: the remedy is the same sentence ("fix the file"), and the + # exception already says which shape it could not move. What matters + # is that it aborts here — the stale-baseline hazard this block exists + # to close is exactly what a swallowed write would leave behind. + # + # The abort still leaves the spec as this re-arm FOUND it, but no longer + # by an undo written here: the transaction guard around this whole window + # rolls the spec back on every fault that escapes it, so this arm only has + # to raise. Without that rollback a spec with a movable `status:` beside an + # unmovable `baseline_revision:` came back flipped to the re-drive's status + # and stripped of the terminal result, while the run still called the story + # escalated. + raise RearmError( + f"cannot re-stamp baseline_revision on {spec_path} " + f"({e.__class__.__name__}: {e}) — fix the file, then re-run resolve" + ) from e + if overwritten and overwritten != old_baseline: + # Compared against `old_baseline` — what the RUN recorded for the + # escalated attempt — NOT against `task.baseline_commit`, which the + # advance above has already moved to the new HEAD. Measuring against the + # advanced value made this fire on every ordinary from-scratch re-arm + # whose resolve session committed anything: the spec and the run agreed + # exactly, and the operator was still told they diverged. A record that + # fires on the routine case is the "trains the operator to scroll past + # the meaningful one" failure the `restore` split exists to prevent. + # + # What survives is the real signal, on BOTH legs: the spec claimed a + # baseline the run never recorded. That is the only trace left of a + # divergence the gate can no longer report, because the re-stamp is + # about to normalize it away. + journal.append( + "rearm-baseline-restamped", + story_key=key, + spec_file=str(spec_path), + overwritten=overwritten, + baseline=task.baseline_commit, + restore=bool(restore_patch), + ) - save_state(run_dir, state) + save_state(run_dir, state) + except BaseException as e: + # Roll the spec back to the bytes this re-arm found, record that it aborted, and + # re-raise the ORIGINAL fault. A failed rollback raises out of here instead — + # a part-written spec is the loudest thing this can be, and the original fault + # rides along in that `RearmError`'s `__context__` because it is still being + # handled at the moment the restore raises. + # + # `BaseException` and not `Exception`, and the breadth is load-bearing rather + # than defensive: `KeyboardInterrupt` and `SystemExit` derive from `BaseException` + # alone, and this window spends most of its time in blocking I/O an operator can + # interrupt — three git subprocesses (`rev_parse_head`, `untracked_files`, + # `commits_above`) plus `save_state`, all AFTER the status flip has published and + # BEFORE anything persists it. A Ctrl-C there under `except Exception` would exit + # by the one path that reproduces exactly the DW-79/DW-83 state this guard exists + # to end: a spec re-armed on disk against a task still recorded as ESCALATED. + # Narrowing this arm is a silent regression, so a test raises `KeyboardInterrupt` + # through the window on purpose. + # + # That breadth is also what makes the commit point AMBIGUOUS on exactly one path, + # and the check below is the price of it: `save_state` commits by `atomic_replace` + # and can still be interrupted between that rename and its return, so a fault + # arriving here does NOT prove the transaction failed. `_rearm_commit_landed` asks + # the disk — the only witness of a rename — and a committed re-arm is left alone: + # undoing the spec then would build the mirror image of the defect this guard + # closes, persisted state re-armed against a spec that is not. + if not _rearm_commit_landed(run_dir, key, task): + _rollback_rearm(journal, key, spec_path, spec_before, task, state, e) + raise journal.append( "story-escalation-resolved", story_key=key, @@ -4425,6 +4766,34 @@ def rearm_event_notice( "abandoned attempt rather than your resolve, revert them now", "", ) + if kind == "rearm-commits-probe-failed": + # The sibling row above is written only when the probe ANSWERED, so its + # absence used to mean either "no commits from the abandoned attempt" or + # "the probe could not tell" — and nothing downstream could separate them. + # The range is named in BOTH halves on purpose: `resolve` prints the + # next_step and the TUI drops it, so the message has to stand alone there. + baseline = str(entry.get("old_baseline", "?")) + # A first-party value is a full rev-parse hex result. Refuse to turn a + # malformed persisted value into a copy/paste command, and collapse control + # characters from git's detail before either operator surface renders it. + base = baseline[:12] if re.fullmatch(r"[0-9a-fA-F]{12,}", baseline) else "unknown" + detail = str(entry.get("error", "?")) + if baseline: + # `commits_above` repeats the complete baseline in its GitError. The + # display label and command deliberately use the short form, so leaving + # the full value in the detail defeated that truncation on both surfaces. + detail = detail.replace(baseline, base) + detail = re.sub(r"[\x00-\x1f\x7f\ud800-\udfff]+", " ", detail) + if len(detail) > 4096: + detail = detail[:4093] + "..." + return ( + "warning", + f"could not list the commits above the abandoned attempt's baseline " + f"({base}..) — {detail}; this silence proves nothing, so fix the Git " + f"failure, then run `git log {base}..HEAD` yourself and revert anything " + "that did not come from your resolve", + f"Fix the Git failure, then check `git log {base}..HEAD` before resuming", + ) if kind == "rearm-baseline-advance-failed": return ( "warning", @@ -4559,6 +4928,73 @@ def rearm_event_notice( "that divergence", "", ) + if kind == "rearm-aborted": + # ONE kind, THREE renderings, told apart by `rollback` — a field the producer + # writes because this reader runs out of process and cannot look at the spec to + # see what is on disk. The split is by what the surface may CLAIM about the file, + # not by how the re-arm failed: + # + # * `restored` / `unchanged` — `_restore_rearmed_spec` either put a landed write + # back or READ the file and proved it byte-equal. Both license "left exactly as + # the re-arm found it", so they share a message. + # * `failed` — the restore could not write, so the spec may be half-written and + # no re-run of resolve can settle it. + # * anything else — `unknown`, an absent field, or a value this table does not + # recognize. Says what is TRUE regardless (nothing persisted, still escalated) + # and claims nothing about the file. The default is deliberately the + # non-reassuring branch: an unknown outcome rendered as the benign one is how a + # sentinel-clear abort came to describe a DELETED file as untouched, and a + # record written by a future producer must not inherit a reassurance by + # accident. + # + # Position-independent wording, because the same string renders as a `resolve` + # stderr line and as a TUI toast, and the TUI drops the `next_step`: the message + # alone has to carry everything an operator must act on. That is why `failed` + # names the restore-from-git remedy in the MESSAGE and keeps it in `next_step` + # too — this is the one re-arm kind whose imperative is not moot on the TUI, + # since an abort raises and that surface does not go on to resume. + # + # Neither surface may read this as a re-arm that half-succeeded — an abort raises, + # so `rearm_holds_the_resume` is deliberately NOT extended to this kind. There is + # no gesture left to hold. + spec = entry.get("spec_file", "") or "(none)" + error = entry.get("error", "?") + rollback = str(entry.get("rollback", "")) + if rollback == "failed": + # No enumeration of WHICH writes landed. A fault raised inside + # `strip_auto_run_result` reaches the guard with the flip published and the + # `## Auto Run Result` section still present, so the old sentence ("carrying + # this re-arm's status flip and missing its `## Auto Run Result` section") + # described a state this record cannot know it is in. + # + # Nor does it promise GIT alone. The bytes the undo failed to write lived only + # in this process and are gone with it, and a spec is not necessarily tracked: + # an untracked artifact, or one in an artifacts folder configured outside the + # checkout entirely (supported configuration — `bmadconfig` resolves one), has + # no committed copy to check out. Naming git as THE remedy sent that operator + # to a command with nothing to give them; naming it as ONE of two keeps the + # common case one word away without asserting a recovery that may not exist. + return ( + "warning", + f"the re-arm ABORTED ({error}) and putting the spec back FAILED — {spec} " + "may be left part-written, so restore it from git or from your own copy " + "before re-running resolve; nothing was persisted and the story is still " + "escalated", + "Restore the spec from git or your own copy, then re-run resolve", + ) + if rollback in ("restored", "unchanged"): + return ( + "warning", + f"the re-arm ABORTED ({error}) — nothing was persisted, the spec ({spec}) " + "was left exactly as the re-arm found it, and the story is still escalated", + "Fix the cause above, then re-run resolve", + ) + return ( + "warning", + f"the re-arm ABORTED ({error}) — nothing was persisted and the story is still " + f"escalated, but the re-arm could not confirm what it left on disk ({spec})", + "Check the recorded spec, then re-run resolve", + ) return None @@ -4588,11 +5024,15 @@ def rearm_holds_the_resume(entry: dict[str, Any]) -> bool: but futile: the re-drive re-plans from a tree that never saw the correction and mints the same sentinel again. - The other warnings stay advisory and do NOT hold. `stale-restore-commits`, - `stale-restore-unparseable` and `rearm-baseline-advance-failed` each report - something an operator may need to act on, but none of them PROVES the re-drive - cannot route, and holding on a maybe would turn the ordinary degrade path into a - two-command gesture for an outcome nothing decided. + The other warnings stay advisory and do NOT hold — `stale-restore-commits`, + `stale-restore-unparseable`, `rearm-commits-probe-failed` and + `rearm-baseline-advance-failed` among them: each reports something an operator may + need to act on, but none of them PROVES the re-drive cannot route, and holding on a + maybe would turn the ordinary degrade path into a two-command gesture for an + outcome nothing decided. `rearm-commits-probe-failed` is the newest and the least + tempting to promote: it says the commits probe could not answer, which is strictly + LESS than the answer — it proves nothing about whether the re-drive can route, only + that one advisory could not be computed. Not folded into `rearm_event_notice`'s tuple, because they are different questions asked of the same entry: that table answers "what do I tell the operator", this @@ -4629,8 +5069,11 @@ def _stale_restore_residue( human is the classifier. `bmad-loop resolve` echoes these to stderr. Best-effort throughout: a deleted or unreadable patch, a non-repo project, a - bad old baseline — none may wedge a resolve. A patch parse failure journals - its degrade; a commits-probe Git failure deliberately degrades silently. + bad old baseline — none may wedge a resolve. Both degrades journal a record of + their own: a patch parse failure writes `stale-restore-unparseable`, and a + commits-probe Git failure writes `rearm-commits-probe-failed` (DW-81). Neither + is silent, because the sibling record's ABSENCE is what an operator reads as + "clean" — and a probe that could not answer is not the same claim. """ if not old_latch: return set() @@ -4661,17 +5104,34 @@ def _stale_restore_residue( if old_baseline: try: shas = verify.commits_above(repo, old_baseline) - except verify.GitError: - # Follow rearm_escalation's baseline-advance taxonomy boundary; - # this warn-only probe remains silent. - shas = [] - if shas: + except verify.GitError as e: + # Follows rearm_escalation's baseline-advance taxonomy boundary, and the + # catch stays EXACTLY `verify.GitError`: every fault `_run_git` can + # translate (spawn, timeout, decode, non-zero rc) lands here, and a + # non-git fault still escapes to the re-arm's transaction guard. + # + # Warn-only, but no longer silent. This arm used to set `shas = []` and + # fall through to the `if shas:` gate below, which made a probe that + # FAILED byte-identical — on every downstream surface — to a probe that + # found no commits above the old baseline. The operator was told nothing + # either way, and the absent record is the only warning they get that the + # abandoned attempt's commits may still be sitting under the re-drive's + # new baseline. The `else:` is what retires the sentinel rather than + # leaving it inert: no `shas` exists on this leg to gate on. journal.append( - "stale-restore-commits", + "rearm-commits-probe-failed", story_key=story_key, old_baseline=old_baseline, - commits=shas, + error=f"{e.__class__.__name__}: {e}", ) + else: + if shas: + journal.append( + "stale-restore-commits", + story_key=story_key, + old_baseline=old_baseline, + commits=shas, + ) return residue diff --git a/src/bmad_loop/sweep.py b/src/bmad_loop/sweep.py index b3947afe..bc66cb07 100644 --- a/src/bmad_loop/sweep.py +++ b/src/bmad_loop/sweep.py @@ -543,10 +543,15 @@ def _rearm_generation(task: StoryTask) -> None: what makes the next dispatch re-mint ``attempt == 1`` — an id byte-equal to the abandoned attempt's, since ``engine._session_task_id`` emits its discriminator only above zero. The artifact a shared id corrupts is ``tasks//escalation.json``: the - sweep skill writes it and ``resolve._gather_escalations`` reads it once per RECORDED - session, so two records carrying one id return the abandoned cycle's escalation for - the fresh session too. ``result.json`` is NOT at risk: both adapters unlink it in - ``start_session``. + sweep skill writes it, and two records carrying one id both name that one mutable + file, so the abandoned cycle's escalation is the fresh session's too. + ``resolve._gather_escalations`` now opens each distinct ``task_id`` once and + de-duplicates entries by content, so it no longer reports the same aliased file + twice. Both adapters also unlink cycle outputs in ``start_session``, which stops a + healthy restart from inheriting stale contents — but cleanup still leaves the two + historical records naming one mutable directory: a healthy restart erases the + abandoned cycle's artifact, while a re-escalation replaces it for both records. + Minting a fresh id is what preserves one artifact namespace per recorded cycle. Same pattern as ``runs.rearm_escalation``, DIFFERENT reason: #705's harm is ``_resumable_session`` verdict replay, which runs only on the dev/review phases and diff --git a/src/bmad_loop/tui/app.py b/src/bmad_loop/tui/app.py index b64cc1ff..2b59c5a8 100644 --- a/src/bmad_loop/tui/app.py +++ b/src/bmad_loop/tui/app.py @@ -969,7 +969,19 @@ def _do_rearm( before_entries = runs.journal_entries_or_none(run_dir) hold_resume = False try: - runs.rearm_escalation(run_dir, story_key, isolated_redrive=isolation == "worktree") + runs.rearm_escalation( + run_dir, + story_key, + isolated_redrive=isolation == "worktree", + # DW-11. This gesture runs no resolve session, so it accepted nothing: + # the escalation watermark must not advance. A `resolution.json` on + # disk is NOT evidence to the contrary here — `_restore_recorded` + # already records the governing fact for this surface, that a stale + # marker is indistinguishable from a fresh one, which is why this path + # declines the restore latch too. Stamping on its presence would bury + # escalations raised since the marker was written. + resolution_recorded=False, + ) except RearmError as e: self.notify(f"re-arm failed: {e}", severity="error") return @@ -977,7 +989,8 @@ def _do_rearm( # In the `finally`, matching `cli.cmd_resolve`. `_stale_restore_residue` # journals BEFORE the re-stamp block that raises `RearmError`, so on that # path the records were already written and returning early threw them - # away — including `stale-restore-commits`, the one record whose whole + # away — including the commits PAIR (`stale-restore-commits` when the probe + # answered, `rearm-commits-probe-failed` when it could not), whose whole # point is that nothing else will tell the human. This surface used to # `return` there while the CLI echoed, so the two DID drift on the abort # path even after they were unified on routing — and an abort is when the diff --git a/tests/conftest.py b/tests/conftest.py index 159097b5..340b266a 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -153,6 +153,26 @@ def force_psmux_backend(monkeypatch): multiplexer.get_multiplexer.cache_clear() +def json_recursion_payload() -> str: + """A nested value the real decoder cannot parse without recursing too deep. + + The depth is discovered rather than hardcoded. 3.13 raises at ``limit * 20``; + 3.14 decodes that iteratively and only recurses far deeper, where the + threshold follows the C stack rather than ``sys.getrecursionlimit()``. + Probing keeps callers regression tests for a genuine ``RecursionError`` + rather than a synthetic stand-in. + """ + limit = sys.getrecursionlimit() + for multiplier in (20, 100, 500, 2000): + depth = limit * multiplier + payload = "[" * depth + "0" + "]" * depth + try: + json.loads(payload) + except RecursionError: + return payload + pytest.skip("the json decoder does not recurse at any probed depth on this interpreter") + + def write_script_launcher(directory: Path, name: str, body: str) -> Path: """Write a fake CLI launcher for the host OS.""" directory = Path(directory) diff --git a/tests/test_cli.py b/tests/test_cli.py index a4e4779c..d70d8c0e 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -2624,7 +2624,9 @@ def test_resolve_restamps_the_code_root_before_it_rearms(project, monkeypatch, c run_dir, moved, _ = _resolve_run_with_a_moved_code_root(project, monkeypatch) seen: list = [] - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): seen.append(load_state(rd).code_root) return key @@ -2748,7 +2750,9 @@ def test_resolve_echoes_this_rearms_stale_restore_events(tmp_path, monkeypatch, run_dir = _escalated_run(tmp_path, "r1") Journal(run_dir).append("stale-restore-excluded", story_key="s1", files=["FROM-LAST-TIME.txt"]) - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): journal = Journal(rd) journal.append("stale-restore-excluded", story_key=key, patch="a.patch", files=["new.txt"]) journal.append("stale-restore-unparseable", story_key=key, patch="b.patch", error="OSErr") @@ -2788,7 +2792,9 @@ def test_resolve_echoes_the_rearm_baseline_records(tmp_path, monkeypatch, capsys _escalated_run(tmp_path, "r1") - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): journal = Journal(rd) journal.append( "rearm-baseline-advance-failed", @@ -2839,7 +2845,9 @@ def test_resolve_restamp_echo_warns_on_both_legs(tmp_path, monkeypatch, capsys): from bmad_loop.journal import Journal def rearm_with(restore: bool): - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): Journal(rd).append( "rearm-baseline-restamped", story_key=key, @@ -2893,7 +2901,9 @@ def test_resolve_survives_a_corrupt_journal(tmp_path, monkeypatch, capsys, outco from bmad_loop import runs from bmad_loop.journal import JOURNAL_FILE - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): if outcome == "rearm-error": raise runs.RearmError("cannot re-open story spec /x/spec.md") return key @@ -2929,7 +2939,9 @@ def test_resolve_echoes_a_skipped_restamp(tmp_path, monkeypatch, capsys): _escalated_run(tmp_path, "r1") - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): Journal(rd).append( "rearm-baseline-restamp-skipped", story_key=key, @@ -2950,20 +2962,37 @@ def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): def test_resolve_echoes_the_residue_even_when_the_rearm_aborts(tmp_path, monkeypatch, capsys): - """An abort is when the residue matters MOST, so the echo lives in a `finally`. - - `runs._stale_restore_residue` journals BEFORE the re-stamp block that raises - `RearmError`, so on that path the records are already on disk when the abort - happens — and an echo placed after an early `return 1` threw away records the - re-arm had genuinely written. The one it threw away is the one that cannot be - recovered from anywhere else: `stale-restore-commits` names commits now sitting - below a baseline the operator is looking at in a half-run re-arm, and nothing - but this line will tell them. The failure and the residue are both true, and the - operator needs both to decide what to do with the tree. + """An abort is when the residue matters MOST, so the echo lives in a `finally` — and + since the whole re-arm window became one transaction, the echo has to say the re-arm + ABORTED as well as what it left behind (DW-85). + + `runs._stale_restore_residue` journals BEFORE anything that can raise past it, so on + an abort those records are already on disk when the fault happens — and an echo + placed after an early `return 1` threw away records the re-arm had genuinely written. + The one it threw away is the one that cannot be recovered from anywhere else: + `stale-restore-commits` names commits now sitting below a baseline the operator is + looking at, and nothing but this line will tell them. + + But that line alone MISDESCRIBES the tree once the rollback exists. It says files + were "excluded from the re-drive baseline" and commits "sit below the re-drive's new + baseline" — a baseline `save_state` never persisted, because the transaction rolled + the whole window back. The residue records are true observations of what the re-arm + LOOKED at and false as a description of what it LEFT, so `rearm-aborted` is journalled + from the rollback and echoed beside them: the operator needs both, and needs to know + which one describes the disk. + + The fake journals the two records in the order the real path writes them (the residue + pass, then the abort record from `_rollback_rearm`'s `finally`) and then raises, which + is what makes the echo's one walk over the new entries the thing under test. Ablation (residue echo): move `_echo_rearm_events` out of the `finally` back under the `try` and the commits assertion reddens while the `error:` line still prints. + Ablation (abort line): drop the `rearm-aborted` arm from `runs.rearm_event_notice` + and the "still escalated" assertion reddens while the commits line still prints — + which is exactly the DW-85 state, a true residue notice with nothing saying it + describes a baseline that was never saved. + Ablation (success output): deleting the gate outright does NOT grade the last assertion. Drop `return 1` from `cmd_resolve`'s `except runs.RearmError` arm and the success line does leak to stdout, but `main` then answers 0 and the exit-code @@ -2977,11 +3006,21 @@ def test_resolve_echoes_the_residue_even_when_the_rearm_aborts(tmp_path, monkeyp _escalated_run(tmp_path, "r1") - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): # journalled first, exactly as the real residue pass is ordered Journal(rd).append( "stale-restore-commits", story_key=key, old_baseline="f" * 40, commits=["c1", "c2"] ) + # ...and then the rollback's own record, from `_rollback_rearm`'s `finally` + Journal(rd).append( + "rearm-aborted", + story_key=key, + spec_file="/p/specs/s1.md", + error="OSError: [Errno 28] No space left on device", + rollback="restored", + ) raise runs.RearmError("could not re-stamp the spec baseline") monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) @@ -2993,6 +3032,11 @@ def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): out, err = capsys.readouterr() assert "error: could not re-stamp the spec baseline" in err # the abort still reports assert "2 commit(s) sit below the re-drive's new baseline (ffffffffffff..)" in err + # ...and the line that says the baseline those commits sit below was never persisted + assert "the re-arm ABORTED" in err + assert "nothing was persisted" in err + assert "still escalated" in err + assert "/p/specs/s1.md" in err assert "re-armed" not in out # ...and the failure is not dressed up as a success @@ -3032,7 +3076,9 @@ def test_resolve_holds_the_resume_when_the_correction_cannot_reach_the_redrive( from bmad_loop.journal import Journal def rearm_journalling(kind, **fields): - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): Journal(rd).append(kind, story_key=key, **fields) return key @@ -3099,7 +3145,9 @@ def test_resolve_appends_the_next_step_imperative(tmp_path, monkeypatch, capsys) _escalated_run(tmp_path, "r1") - def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): journal = Journal(rd) journal.append( # table row with a next_step "rearm-baseline-advance-failed", @@ -3127,6 +3175,55 @@ def fake_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): assert commits.endswith("revert them now") +def test_resolve_echoes_the_commits_probe_failure(tmp_path, monkeypatch, capsys): + """The failed commits probe reaches stderr, imperative and all (DW-81). + + Before it had a record, a probe that FAILED and a probe that found nothing were + byte-identical here: neither journalled, so neither printed, and the operator was + never told that the abandoned attempt's commits might be sitting under the + re-drive's new baseline unreverted. Graded on this surface specifically because + it is the one that appends `next_step` — the TUI drops it — and because the + record is warn-only by contract, so an echo is the only place it can ever appear. + + Ablation: delete the `rearm-commits-probe-failed` row from + `runs.rearm_event_notice` and the line is not printed at all, so `next(...)` + raises `StopIteration`; drop `tail` from `_echo_rearm_events`' f-string and only + the `endswith` reddens. + """ + from bmad_loop import runs + from bmad_loop.journal import Journal + + _escalated_run(tmp_path, "r1") + baseline = "b" * 40 + + def fake_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): + Journal(rd).append( + "rearm-commits-probe-failed", + story_key=key, + old_baseline=baseline, + error=f"GitError: git rev-list {baseline}..HEAD failed in /code: " + "not a git repository", + ) + return key + + monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) + monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--resume"]) == 0 + ) + + lines = capsys.readouterr().err.splitlines() + probe = next(ln for ln in lines if "could not list the commits above" in ln) + assert probe.startswith("warning: ") # advisory severity, rendered as the prefix + assert "bbbbbbbbbbbb.." in probe and "b" * 40 not in probe # truncated, not raw + assert "not a git repository" in probe # the typed cause survives the echo + assert probe.endswith( + "; Fix the Git failure, then check `git log bbbbbbbbbbbb..HEAD` before resuming" + ) + + def test_resolve_interactive_runs_session_then_rearms(tmp_path, monkeypatch): from bmad_loop import resolve from bmad_loop.journal import load_state @@ -3135,7 +3232,9 @@ def test_resolve_interactive_runs_session_then_rearms(tmp_path, monkeypatch): _escalated_run(tmp_path, "r1") calls = {} monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: calls.setdefault("ctx", True)) + monkeypatch.setattr( + resolve, "build_context", lambda *a, **k: (calls.setdefault("ctx", True), 0, 0) + ) monkeypatch.setattr( resolve, "run_session", lambda *a, **k: calls.setdefault("session", True) or True ) @@ -3176,7 +3275,7 @@ def fake_session(adapter, project, rd, story_key, *, generation, model=""): return True monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) monkeypatch.setattr(resolve, "run_session", fake_session) # --no-resume: re-arm only, so the bump this row contrasts against still runs assert cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-resume"]) == 0 @@ -3190,7 +3289,14 @@ def test_resolve_interactive_unsupported_adapter(tmp_path, monkeypatch, capsys): _escalated_run(tmp_path, "r1") monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + # DW-11: a NON-ZERO withheld count, deliberately. This command is about to fail, + # and an operator must not be told escalations were withheld from an agent that + # never launched — which is why the count is printed AFTER the adapter has proved + # it supports an interactive session, not beside the context build. + # + # Ablation: move the withheld print above the `try:` and this row reddens on the + # stdout assertion below. + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 3, 0)) def boom(*a, **k): raise NotImplementedError @@ -3198,7 +3304,388 @@ def boom(*a, **k): monkeypatch.setattr(resolve, "run_session", boom) rc = cli.main(["resolve", "--project", str(tmp_path), "r1"]) assert rc == 1 - assert "no interactive session mode" in capsys.readouterr().err + captured = capsys.readouterr() + assert "no interactive session mode" in captured.err + assert "were not shown" not in captured.out + + +def _withheld_line(out: str) -> str: + (line,) = [ln for ln in out.splitlines() if "were not shown" in ln] + return line + + +def test_resolve_reports_the_escalations_it_withheld(tmp_path, monkeypatch, capsys): + """The number an operator reads is `build_context`'s OWN second member, not a + constant and not a re-derivation. Seeded to 3 so a hardcoded 1 (or a length of + something else) cannot pass, and worded for what the code can prove: these entries + were PRESENTED to an earlier cycle that recorded a resolution. + + Ablation: delete the `if withheld:` print from `cmd_resolve` and this reddens.""" + from bmad_loop import resolve + + _escalated_run(tmp_path, "r1") + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 3, 0)) + monkeypatch.setattr(resolve, "run_session", lambda *a, **k: True) + + assert cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-resume"]) == 0 + + line = _withheld_line(capsys.readouterr().out) + assert line.startswith("3 earlier escalation(s) for s1 were not shown") + assert "recorded a resolution" in line + + +def test_resolve_says_nothing_when_it_withheld_nothing(tmp_path, monkeypatch, capsys): + """A first cycle, and every pre-upgrade `state.json`, withholds nothing — and must + print nothing, or the line becomes noise on the surface it exists to inform. + + `launching resolve agent` is the positive control: an absence assertion passes for + every reason stdout could be empty, including a command that returned before it + ever reached the print. + + Ablation: make the print unconditional (drop `if withheld:`) and this reddens.""" + from bmad_loop import resolve + + _escalated_run(tmp_path, "r1") + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) + monkeypatch.setattr(resolve, "run_session", lambda *a, **k: True) + + assert cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-resume"]) == 0 + + out = capsys.readouterr().out + assert "launching resolve agent for s1" in out # the path WAS taken + assert "were not shown" not in out + + +def test_resolve_no_interactive_builds_no_context_and_reports_nothing( + tmp_path, monkeypatch, capsys +): + """`--no-interactive` runs no agent, so there is no context to filter and no + audience for the count. It also accepted nothing IN THIS GESTURE, so the watermark + must stand — the human may have fixed the spec by hand, but nothing recorded which + escalations that answered. The generation bump is the positive control that the + re-arm really ran. + + The run carries a session record deliberately: on a task with an EMPTY `sessions` + list an unconditional stamp writes `len([]) == 0`, so `escalations_resolved_upto == + 0` would hold with the gate ablated and the assertion would grade nothing.""" + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, "r1", details=("never answered",)) + built: list[int] = [] + monkeypatch.setattr( + resolve, "build_context", lambda *a, **k: (built.append(1), (None, 5, 0))[1] + ) + + assert ( + cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-interactive", "--no-resume"]) + == 0 + ) + + assert built == [] + assert "were not shown" not in capsys.readouterr().out + task = load_state(run_dir).tasks["s1"] + assert len(task.sessions) == 1 # a stamp here would be a VISIBLE 1 + assert task.escalations_resolved_upto == 0 + assert task.generation == 1 # positive control: the re-arm ran + + +def _escalated_trail_run(tmp_path, run_id="r1", *, details=("first cycle",)): + """An escalated run whose task carries one completed session record per entry in + `details`, each with the `tasks//escalation.json` the engine wrote when it + paused. Nothing about the escalation walk is stubbed by the rows that use it.""" + import json as _json + + from bmad_loop.engine import _session_task_id + from bmad_loop.journal import load_state, save_state + from bmad_loop.model import SessionRecord + + run_dir = _escalated_run(tmp_path, run_id) + state = load_state(run_dir) + task = state.tasks["s1"] + task.sessions.clear() + for seq, detail in enumerate(details, start=1): + task_id = _session_task_id("s1", "review", seq, 0) + task.record_session(SessionRecord(task_id=task_id, role="dev", status="completed")) + d = run_dir / "tasks" / task_id + d.mkdir(parents=True, exist_ok=True) + (d / "escalation.json").write_text( + _json.dumps({"escalations": [{"severity": "CRITICAL", "detail": detail}]}), + encoding="utf-8", + ) + save_state(run_dir, state) + return run_dir + + +def _redrive_escalates(run_dir, detail): + """What a re-driven session that escalated again leaves behind, re-escalated so a + second `bmad-loop resolve` is legal on it.""" + import json as _json + + from bmad_loop.engine import _session_task_id + from bmad_loop.journal import load_state, save_state + from bmad_loop.model import Phase, SessionRecord + + state = load_state(run_dir) + task = state.tasks["s1"] + task_id = _session_task_id("s1", "review", 1, task.generation) + assert task_id not in {r.task_id for r in task.sessions} + task.record_session(SessionRecord(task_id=task_id, role="dev", status="completed")) + d = run_dir / "tasks" / task_id + d.mkdir(parents=True, exist_ok=True) + (d / "escalation.json").write_text( + _json.dumps({"escalations": [{"severity": "CRITICAL", "detail": detail}]}), + encoding="utf-8", + ) + task.phase = Phase.ESCALATED + save_state(run_dir, state) + + +def _marker_writing_session(run_dir_marker=True): + from bmad_loop import resolve + + def fake_session(adapter, project, rd, story_key, *, generation, model=""): + marker = resolve.resolution_path(rd, story_key) + marker.parent.mkdir(parents=True, exist_ok=True) + if run_dir_marker: + marker.write_text("{}", encoding="utf-8") + return run_dir_marker + + return fake_session + + +def test_resolve_prints_the_number_the_real_walk_produced(tmp_path, monkeypatch, capsys): + """Every other CLI row here stubs `build_context` to a literal, so the number an + operator actually sees is otherwise never produced by the real walk. This row runs + two whole cycles with only `_make_adapters` and `run_session` stubbed: the first + shows both escalations and withholds nothing, the re-arm stamps the watermark, the + re-drive escalates again, and the second cycle prints the count `_gather_escalations` + computed — against a `context.json` that carries only the new entry. + + Ablation: revert `_gather_escalations` to the unsliced walk and the second cycle + prints nothing while `context.json` carries all three.""" + import json as _json + + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, details=("older A", "older B")) + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "run_session", _marker_writing_session()) + + argv = ["resolve", "--project", str(tmp_path), "r1", "--no-resume"] + assert cli.main(argv) == 0 + first = capsys.readouterr().out + assert "launching resolve agent for s1" in first + assert "were not shown" not in first # a first cycle withholds nothing + assert load_state(run_dir).tasks["s1"].escalations_resolved_upto == 2 + + _redrive_escalates(run_dir, "raised by the re-drive") + + assert cli.main(argv) == 0 + assert _withheld_line(capsys.readouterr().out).startswith( + "2 earlier escalation(s) for s1 were not shown" + ) + ctx = _json.loads(resolve.context_path(run_dir, "s1").read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == ["raised by the re-drive"] + + +def test_resolve_withholds_coverage_when_a_session_artifact_could_not_be_read( + tmp_path, monkeypatch, capsys +): + """The whole F1 channel, unstubbed past `_make_adapters` and `run_session`: the + walk skips an unreadable artifact, `build_context` reports the skip, and + `cmd_resolve` refuses to record coverage over it. + + Without the refusal the watermark would stamp `len(task.sessions)` — covering the + session whose escalations this cycle never showed anyone — and the NEXT cycle, + reading the same file cleanly, would withhold them as already answered. The + escalation is then invisible for the rest of the run. + + The second cycle is the proof and the reason a one-cycle assertion is not enough: + it repairs the artifact and re-runs, and the escalation that was skipped comes back + SHOWN rather than counted as withheld. + + `re-armed s1` is the positive control on both cycles — the refusal withholds + coverage, never the re-arm itself. + + Ablation: restore `resolution_recorded = bool(produced)` and this reddens at the + first cycle's watermark (1 != 0), and again on the second cycle's shown list.""" + import json as _json + + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, details=("raised then unreadable",)) + task_id = load_state(run_dir).tasks["s1"].sessions[0].task_id + artifact = run_dir / "tasks" / task_id / "escalation.json" + good = artifact.read_bytes() + artifact.write_bytes(b'{"escalations": [\xff\xfe]}') + + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "run_session", _marker_writing_session()) + argv = ["resolve", "--project", str(tmp_path), "r1", "--no-resume"] + + assert cli.main(argv) == 0 + first = capsys.readouterr() + assert "re-armed s1" in first.out # the re-arm itself was NOT withheld + assert "could not be read" in first.err + assert "NOT recorded as covering" in first.err + task = load_state(run_dir).tasks["s1"] + assert task.escalations_resolved_upto == 0 # nothing claimed + assert task.generation == 1 # positive control: the gesture really re-armed + + # the transient fault clears; the escalation is still there to be answered + artifact.write_bytes(good) + _redrive_escalates(run_dir, "raised by the re-drive") + assert cli.main(argv) == 0 + second = capsys.readouterr() + assert "re-armed s1" in second.out + assert "were not shown" not in second.out # nothing was ever covered, so nothing hides + ctx = _json.loads(resolve.context_path(run_dir, "s1").read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == [ + "raised by the re-drive", + "raised then unreadable", # the one the first cycle could not show + ] + assert load_state(run_dir).tasks["s1"].escalations_resolved_upto == 2 # now covered + + +def test_resolve_says_nothing_about_unreadable_artifacts_when_none_were_skipped( + tmp_path, monkeypatch, capsys +): + """The ordinary run: a clean run-dir prints no coverage warning and records + coverage exactly as before. Without this row the refusal could fire on every + resolve and every existing count row would still pass. + + Ablation: drop the `if produced and unreadable:` guard (print unconditionally) and + this reddens; invert `resolution_recorded` to `bool(produced) and unreadable` and it + reddens on the watermark.""" + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, details=("first cycle",)) + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "run_session", _marker_writing_session()) + + assert cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-resume"]) == 0 + + captured = capsys.readouterr() + assert "launching resolve agent for s1" in captured.out # the path WAS taken + assert "could not be read" not in captured.err + assert load_state(run_dir).tasks["s1"].escalations_resolved_upto == 1 + + +def test_resolve_records_no_coverage_for_a_skip_when_the_agent_wrote_nothing( + tmp_path, monkeypatch, capsys +): + """The warning is gated on `produced` as well as on the skip: with no resolution + recorded the watermark would not have advanced anyway, so telling the operator a + coverage was withheld describes a loss they never had. The `no resolution recorded` + line is the positive control that this IS the abandoned-session path. + + Ablation: drop the `produced and` half of the print guard and this reddens.""" + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, details=("raised",)) + task_id = load_state(run_dir).tasks["s1"].sessions[0].task_id + (run_dir / "tasks" / task_id / "escalation.json").write_bytes(b'{"escalations": [\xff]}') + + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "run_session", _marker_writing_session(run_dir_marker=False)) + + assert cli.main(["resolve", "--project", str(tmp_path), "r1", "--no-resume"]) == 0 + + err = capsys.readouterr().err + assert "no resolution recorded for s1" in err # the abandoned-session path + assert "could not be read" not in err + assert load_state(run_dir).tasks["s1"].escalations_resolved_upto == 0 + + +def test_resolve_reports_the_withheld_count_when_this_cycle_records_nothing( + tmp_path, monkeypatch, capsys +): + """A watermark already standing filters whatever THIS gesture accepts. The two + halves are independent — `withheld` comes from the walk over what an EARLIER cycle + answered, the stamp from what this one did — but no row paired them: the rows that + assert a number run a marker-writing resolver, and the abandoned-session row asserts + the line's ABSENCE at watermark 0. So a print gated on `resolution_recorded`, or a + count recomputed after the stamp, went ungraded. + + Ablation: gate the withheld print on `resolution_recorded` in `cmd_resolve` and this + row reddens on the missing line while every existing count row stays green.""" + import json as _json + + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, details=("older A", "older B")) + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "run_session", _marker_writing_session()) + + argv = ["resolve", "--project", str(tmp_path), "r1", "--no-resume"] + assert cli.main(argv) == 0 # cycle 1 accepts, stamping the watermark at 2 + capsys.readouterr() + _redrive_escalates(run_dir, "raised by the re-drive") + + # cycle 2 walks away without writing `resolution.json`. The marker cycle 1 wrote is + # still on disk — nothing unlinks it at re-arm — which is the state this path opens + # on for real, and the stub does not clear it either. + monkeypatch.setattr(resolve, "run_session", _marker_writing_session(run_dir_marker=False)) + assert cli.main(argv) == 0 + out = capsys.readouterr() + assert _withheld_line(out.out).startswith("2 earlier escalation(s) for s1 were not shown") + assert "no resolution recorded for s1" in out.err + + task = load_state(run_dir).tasks["s1"] + assert task.escalations_resolved_upto == 2 # UNCHANGED by a gesture that accepted none + assert task.generation == 2 # positive control: it still re-armed + ctx = _json.loads(resolve.context_path(run_dir, "s1").read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == ["raised by the re-drive"] + + +def test_resolve_leaves_the_watermark_when_the_agent_wrote_no_resolution( + tmp_path, monkeypatch, capsys +): + """`cmd_resolve` prints "no resolution recorded" and FALLS THROUGH — no `return` — + so an abandoned or crashed resolve session re-arms the story anyway. That gesture + accepted nothing, so it must not advance the watermark: the escalations the agent + walked away from would otherwise be invisible to every later cycle and reported to + the operator as already answered. + + Driven as a whole SECOND cycle through the real walk, because the consequence is + what the next `resolve` shows, not what one field reads. + + Ablation: remove the `if resolution_recorded:` gate in `rearm_escalation` and this + reddens on the watermark, then again on the second cycle's absent line.""" + import json as _json + + from bmad_loop import resolve + from bmad_loop.journal import load_state + + run_dir = _escalated_trail_run(tmp_path, details=("nobody ever answered this",)) + monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) + monkeypatch.setattr(resolve, "run_session", _marker_writing_session(run_dir_marker=False)) + + argv = ["resolve", "--project", str(tmp_path), "r1", "--no-resume"] + assert cli.main(argv) == 0 + assert "no resolution recorded for s1" in capsys.readouterr().err + + task = load_state(run_dir).tasks["s1"] + assert task.escalations_resolved_upto == 0 # UNCHANGED + assert task.generation == 1 # positive control: the re-arm still ran + + _redrive_escalates(run_dir, "raised by the re-drive") + + assert cli.main(argv) == 0 + assert "were not shown" not in capsys.readouterr().out + ctx = _json.loads(resolve.context_path(run_dir, "s1").read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == [ + "raised by the re-drive", + "nobody ever answered this", + ] def test_resolve_in_ctl_session_detaches_before_resume(tmp_path, monkeypatch, capsys): @@ -3431,7 +3918,7 @@ def fake_session(adapter, project, rd, story_key, *, generation, model=""): return True monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) monkeypatch.setattr(resolve, "run_session", fake_session) called: list = [] monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: called.append(rd) or 0) @@ -3615,7 +4102,7 @@ def fake_session(adapter, project, rd, story_key, *, generation, model=""): return True monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) monkeypatch.setattr(resolve, "run_session", fake_session) monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) rc = cli.main(["resolve", "--project", str(tmp_path), "r1", "--resume"]) @@ -3675,12 +4162,14 @@ def fake_session(adapter, project, rd, story_key, *, generation, model=""): seen: list[bool] = [] - def recording_rearm(rd, key, *, restore_patch=None, isolated_redrive=False): + def recording_rearm( + rd, key, *, restore_patch=None, isolated_redrive=False, resolution_recorded=False + ): seen.append(isolated_redrive) return key monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) monkeypatch.setattr(resolve, "run_session", fake_session) monkeypatch.setattr(runs, "rearm_escalation", recording_rearm) monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: 0) @@ -3713,7 +4202,7 @@ def fake_session(adapter, project, rd, story_key, *, generation, model=""): return True monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) monkeypatch.setattr(resolve, "run_session", fake_session) called: list = [] monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: called.append(rd) or 0) @@ -3744,7 +4233,7 @@ def fake_session(adapter, project, rd, story_key, *, generation, model=""): return True monkeypatch.setattr(cli, "_make_adapters", lambda *a, **k: {"dev": object()}) - monkeypatch.setattr(resolve, "build_context", lambda *a, **k: None) + monkeypatch.setattr(resolve, "build_context", lambda *a, **k: (None, 0, 0)) monkeypatch.setattr(resolve, "run_session", fake_session) called: list = [] monkeypatch.setattr(cli, "_resume_paused_run", lambda proj, rd: called.append(rd) or 0) @@ -5056,7 +5545,7 @@ def test_diagnose_json_emits_pure_document(project, capsys): _seed_run(project.project) doc = machine_json(["diagnose", "--project", str(project.project), "--json"], capsys) - assert doc["schema_version"] == diagnostics.SCHEMA_VERSION == 2 + assert doc["schema_version"] == diagnostics.SCHEMA_VERSION == 3 assert doc["runs"], "the document carries the run it resolved" for canary in CANARIES: assert canary not in json.dumps(doc), f"LEAK via CLI: {canary!r}" @@ -5076,7 +5565,7 @@ def test_diagnose_json_out_writes_document_and_keeps_stdout_empty(project, tmp_p assert "written to" in err # the confirmation moved to stderr written = out_file.read_text() doc = json.loads(written) - assert doc["schema_version"] == diagnostics.SCHEMA_VERSION == 2 + assert doc["schema_version"] == diagnostics.SCHEMA_VERSION == 3 assert "```" not in written # no fences in a file written in JSON mode for canary in CANARIES: assert canary not in written, f"LEAK via CLI: {canary!r}" diff --git a/tests/test_diagnostics.py b/tests/test_diagnostics.py index 07aab2ce..01be5c00 100644 --- a/tests/test_diagnostics.py +++ b/tests/test_diagnostics.py @@ -599,12 +599,19 @@ def test_rearm_records_leak_neither_the_code_root_nor_a_spec_name(): assert restamped["overwritten"] != restamped["baseline"] assert restamped["restore"] is False # a plain flag still ships - # The OTHER three kinds `runs.rearm_escalation` journals `spec_file` on. Routing is + # The OTHER four kinds the re-arm family journals `spec_file` on. Routing is # by field NAME, so these ride the same `_JOURNAL_ALIAS_FIELDS` entry as # `rearm-baseline-restamped` and are correct today for free — which is exactly why # they belong in the sweep: the canary is what catches a field added to one of - # these kinds later, and a sweep that covers two of four grades the routing of a + # these kinds later, and a sweep that covers two of five grades the routing of a # record shape nobody re-checks. + # + # `rearm-aborted` is the fifth and the one written by a DIFFERENT function + # (`runs._rollback_rearm`, from the transaction guard's error path) rather than by + # `rearm_escalation` itself — the divergence that made the routing entry's own + # producer note undercount. It carries two fields the others do not: `error`, which + # the free-text drop set reaches, and `rollback`, a literal enum string that is + # declared benign rather than routed and must therefore still ship VERBATIM. siblings = [ diagnostics._scrub_entry( {"ts": 3.0, "kind": kind, "story_key": STORY_KEY, "spec_file": SPEC_ABS, **extra}, @@ -616,11 +623,20 @@ def test_rearm_records_leak_neither_the_code_root_nor_a_spec_name(): ("rearm-spec-write-unreachable", {"target_branch": REARM_BRANCH}), ("rearm-spec-flip-skipped", {"status": "ready-for-dev"}), ("rearm-baseline-restamp-skipped", {"baseline": SHA}), + ( + "rearm-aborted", + {"error": f"OSError: cannot write {HOME_PATH}/spec.md", "rollback": "restored"}, + ), ) ] # every one of them aliases to the SAME alias as the restamped record above: one # spec, one alias, however many kinds carry it - assert [s["spec_file"] for s in siblings] == [alias, alias, alias] + assert [s["spec_file"] for s in siblings] == [alias, alias, alias, alias] + # the abort record's own two fields: the free-text one is dropped (it quotes a host + # path back), the enum one is deliberately NOT aliased — both surfaces read the + # record for `rollback`, so pseudonymizing it would destroy the field's whole point + assert "error" not in siblings[3] and siblings[3]["error_present"] is True + assert siblings[3]["rollback"] == "restored" assert [orig for ns, orig, _a in pseudo.entries() if ns == "spec"] == [SPEC_NAME] # `rearm-spec-write-unreachable` names the branch the re-drive cuts its replacement @@ -644,6 +660,71 @@ def test_rearm_records_leak_neither_the_code_root_nor_a_spec_name(): assert canary not in rendered, f"LEAK: {canary!r}" +def test_the_two_commit_probe_records_alias_one_baseline_to_one_name(): + """`old_baseline` is a 40-hex sha on BOTH of `_stale_restore_residue`'s records, + and routing is by field NAME, so one entry has to cover both kinds (DW-81). + + It was declared benign in `tests/test_portability_guard.py`'s inventory, which is + the misfiling that inventory's own warning describes — "a name carrying a story + key, a branch, a sha, a spec filename, a path, or free text belongs in a + `diagnostics` table instead" — and the second producer is what forced it. + + The two kinds are graded together rather than one standing in for the other, + because the value's whole use is a comparison an operator makes across them: the + probe-failure record says "I could not tell you what sits above this sha" and the + commits record says "these do". Aliasing one spelling and not the other would + destroy that correlation. That is also why the producer was not respelled to the + already-routed `baseline` — same sha, two spellings, two aliases in one dump. + + Ablation: drop `"old_baseline"` from `_JOURNAL_ALIAS_FIELDS` and the test dies at + the `next(...)` alias lookup with `StopIteration`. The canary sweep is not the + grade: depending on its entropy, the fallback may redact a sha as a secret rather + than preserving the correlatable alias this table promises. + + `commits` remains deliberately outside this test and outside DW-81's routing + change. It is a list on `stale-restore-commits` but an integer count on + `rollback-manual-required`; routing it requires a separate, kind-scoped policy. + """ + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + probe_failed = diagnostics._scrub_entry( + { + "ts": 1.0, + "kind": "rearm-commits-probe-failed", + "story_key": STORY_KEY, + "old_baseline": SHA, + "error": f"GitError: git rev-list {SHA}..HEAD failed in {HOME_PATH}", + }, + pseudo, + {}, + 1.0, + ) + commits = diagnostics._scrub_entry( + { + "ts": 2.0, + "kind": "stale-restore-commits", + "story_key": STORY_KEY, + "old_baseline": SHA, + "commits": ["c" * 40], + }, + pseudo, + {}, + 1.0, + ) + + alias = next(a for ns, orig, a in pseudo.entries() if ns == "commit" and orig == SHA) + # aliased, not dropped — the key stays and only the VALUE is replaced + assert probe_failed["old_baseline"] == commits["old_baseline"] == alias != SHA + # one legend entry for the shared baseline, not one per record spelling + assert {orig for ns, orig, _a in pseudo.entries() if ns == "commit"} == {SHA} + # the free-text sibling on the probe record quotes both the sha and a host path + # back, and is reached by the drop set rather than aliased + assert "error" not in probe_failed and probe_failed["error_present"] is True + + rendered = json.dumps([probe_failed, commits]) + for canary in (SHA, PROPRIETARY, HOME_PATH, *CANARIES): + assert canary not in rendered, f"LEAK: {canary!r}" + + def test_sentinel_upstream_record_drops_the_stories_root_it_names(): """`rearm-upstream-write-unreachable` carries an absolute host path naming the folder a sentinel's upstream correction has to land in. @@ -895,6 +976,240 @@ def test_target_field_routes_by_kind_because_it_carries_two_kinds_of_value(): assert canary not in rendered, f"LEAK: {canary!r}" +def test_stranded_bundle_story_keys_are_aliased_element_wise(): + """`sweep-inflight-stranded` carries a LIST of story keys, and a list of + identifier-shaped strings is the one shape `scrub_json` passes through + untouched — `scrub_json(["1-1-acme-auth"]) == ["1-1-acme-auth"]`, verbatim. + + So the plural field needs the same routing as the singular `story_key` beside + it, which was already aliased: `_JOURNAL_KEYLIST_FIELDS` reduces a list + element-wise, and the namespace selection has to send this one to `story` (not + to `dw`, which is only `dw_ids`) or one dump would carry two different aliases + for the same story. The epic lookup rides along, exactly as it does for `keys`. + + Ablation: drop `story_keys` from `_JOURNAL_KEYLIST_FIELDS` and the alias + assertions redden with the raw keys coming back verbatim; flip the namespace + selection back to `"story" if k == "keys" else "dw"` and the cross-field + identity assertion reddens (a `dw-` alias for a story key). + """ + other_key = "3.4-AcmeVaultRotation" + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + scrubbed = diagnostics._scrub_entry( + { + "ts": 2.0, + "kind": "sweep-inflight-stranded", + "story_keys": [STORY_KEY, other_key], + }, + pseudo, + {STORY_KEY: 1, other_key: 3}, + 1.0, + ) + # the SAME story, journalled singular by a neighbouring record, must resolve to + # the same alias — that identity is the whole reason this is aliased not dropped + singular = diagnostics._scrub_entry( + {"ts": 3.0, "kind": "sweep-bundle-recovered", "story_key": STORY_KEY}, + pseudo, + {STORY_KEY: 1}, + 1.0, + ) + + assert scrubbed["story_keys"] == [singular["story_key"], scrubbed["story_keys"][1]] + assert STORY_KEY not in scrubbed["story_keys"] + assert other_key not in scrubbed["story_keys"] + assert scrubbed["story_keys"][0] != scrubbed["story_keys"][1] + # the epic lookup still applies: `Pseudonymizer.alias` prefixes a story alias + # with `s`, so each element carries the epic it was looked up under — + # drop the `epic=` argument from the keylist branch and both prefixes become a + # bare `story-` + assert [a.split("-")[0] for a in scrubbed["story_keys"]] == ["s1", "s3"] + # …and nothing landed in the deferred-work namespace, which is where the old + # `"story" if k == "keys" else "dw"` selection would have put both of them + assert not [orig for ns, orig, _a in pseudo.entries() if ns == "dw"] + + rendered = json.dumps([scrubbed, singular]) + for canary in (STORY_KEY, other_key, *CANARIES): + assert canary not in rendered, f"LEAK: {canary!r}" + + +def test_scalar_story_keys_fails_closed_instead_of_shipping_verbatim(): + """`_JOURNAL_KEYLIST_FIELDS` routing was gated on `isinstance(v, list)`, so a + SCALAR value on one of those names fell straight through to `scrub_json` — which + is the identity on an identifier-shaped string. `story_keys="1-1-acme-auth"` came + back verbatim. + + Every producer passes a list today, so this is latent rather than live. That is + the argument FOR closing it rather than against: the routing decision would + otherwise rest on a survey of producers staying true, and the neighbouring + `story_keys` row above is graded on lists only, so nothing would notice. + + Asserted on the raw value's ABSENCE, not on the presence key alone — a + presence-key assertion passes for every reason a value could be missing, + including the field never having been read. + + Ablation: restore the `and isinstance(v, list)` gate on the `elif` and the + absence assertions redden with the raw key coming back.""" + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + scrubbed = diagnostics._scrub_entry( + {"ts": 2.0, "kind": "sweep-inflight-stranded", "story_keys": STORY_KEY}, + pseudo, + {STORY_KEY: 1}, + 1.0, + ) + + assert "story_keys" not in scrubbed, "the unknown-shaped value survived under its own name" + assert scrubbed["story_keys_present"] is True + rendered = json.dumps(scrubbed) + assert STORY_KEY not in rendered + for canary in CANARIES: + assert canary not in rendered, f"LEAK: {canary!r}" + + +def test_off_schema_preference_escalation_keys_are_collapsed_to_presence(): + """`engine._review_and_commit` splats `escalation.preference_escalations(rj)` + into `journal.append`, and those entries come out of a session's own + `result.json` — so an LLM chooses the journal FIELD NAMES. No by-name table can + route a name nobody can enumerate, and `scrub_json` is the identity on an + identifier-shaped scalar, so `customer="AcmeVault"` shipped byte-identical into + a dump whose module docstring ends "the dump will be posted publicly". + + `_JOURNAL_KIND_SCHEMAS` declares the record to be `{type, severity, detail}` and + collapses everything else on that kind. Both halves are graded here: the + off-schema value must be GONE, and the declared fields must NOT be — a policy + that flattened the whole record would pass an absence-only assertion while + destroying the field the record is read for. + + ACCEPTED RESIDUAL, asserted so it stays honest rather than drifting: the key + NAME still reaches the dump as `_present`. That was decided on 2026-08-30 + over a name-free `unrouted_field_count` collapse; this row PINS it, so a future + reader finds it recorded as a decision rather than re-discovering it as a bug. + + Ablation: drop the `preference-escalation` row from `_JOURNAL_KIND_SCHEMAS` and + the `AcmeVault` absence assertions redden.""" + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + scrubbed = diagnostics._scrub_entry( + { + "ts": 2.0, + "kind": "preference-escalation", + "story_key": STORY_KEY, + "type": "preference", + "severity": "MEDIUM", + "detail": "CANARY_ESCALATION prose about " + PROPRIETARY, + "customer": "AcmeVault", + }, + pseudo, + {STORY_KEY: 1}, + 1.0, + ) + + # the off-schema key: collapsed, and its VALUE gone from the entry entirely + assert "customer" not in scrubbed + assert scrubbed["customer_present"] is True + assert "AcmeVault" not in json.dumps(scrubbed), "LEAK: off-schema preference value" + # the declared schema is NOT collapsed — these are why the record is read + assert scrubbed["type"] == "preference" + assert scrubbed["severity"] == "MEDIUM" + # `detail` is in the schema but `_JOURNAL_DROP_FIELDS` reaches it first, which is + # the intended precedence: a stricter table always wins over this one + assert "detail" not in scrubbed and scrubbed["detail_present"] is True + # the entry stays correlatable — the kind policy replaces the FALLBACK only, so + # every routing rule above it still runs + assert scrubbed["story_key"] != STORY_KEY and scrubbed["story_key"].startswith("s1-") + + rendered = json.dumps(scrubbed) + for canary in (STORY_KEY, *CANARIES): + assert canary not in rendered, f"LEAK: {canary!r}" + + +def test_self_minted_fields_survive_a_declared_schema_kind(): + """`Journal.append` stamps `log_task`/`log_pos` onto EVERY entry with + `setdefault`, including one whose kind carries a declared schema. They are + engine-authored, not LLM-authored, so the fail-closed arm must not touch them. + + It did: `log_pos` is outside `{type, severity, detail}`, so a real + `preference-escalation` rendered `log_pos_present: true` and the pane-log byte + offset was gone — on exactly the records an operator opens a dump to trace. + `log_task` was never affected, since aliasing reaches it first; `log_pos` was the + only casualty, which is why a test naming the pair would have stayed green. + + The exemption reads `journal.SELF_MINTED_FIELDS` rather than restating the pair, + so it cannot drift from the `setdefault` calls that create the fields. + + Ablation: drop the `k not in SELF_MINTED_FIELDS` clause from `_scrub_entry`'s + fail-closed arm and the integer assertion reddens with `log_pos_present`.""" + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + scrubbed = diagnostics._scrub_entry( + { + "ts": 2.0, + "kind": "preference-escalation", + "log_task": STORY_KEY, + "log_pos": 4096, + "type": "preference", + "customer": "AcmeVault", + }, + pseudo, + {STORY_KEY: 1}, + 1.0, + ) + + # the byte offset survives as an INTEGER — the whole point of the exemption + assert scrubbed["log_pos"] == 4096 + assert "log_pos_present" not in scrubbed + # the pane-log task pointer is still aliased, not dropped and not raw + assert scrubbed["log_task"] != STORY_KEY and scrubbed["log_task"].startswith("s1-") + # ...while the LLM-authored key on the same record is still collapsed, so the + # exemption did not widen into a general escape from the fail-closed arm + assert "customer" not in scrubbed and scrubbed["customer_present"] is True + + rendered = json.dumps(scrubbed) + for canary in (STORY_KEY, *CANARIES): + assert canary not in rendered, f"LEAK: {canary!r}" + + +def test_decision_pending_question_is_dropped_not_scrubbed(): + """`sweep.py` journals `question=decision.question` on `decision-pending`, and it + was declared benign. A MULTI-WORD question does collapse — but only by accident + of `_IDENTIFIER_RE` forbidding spaces, which is not a property to route on. A + ONE-TOKEN question is identifier-shaped and shipped verbatim. + + So it joins `detail`/`reason`/`blocker`/`suggestion`/`note` in + `_JOURNAL_DROP_FIELDS`, under the same free-text rule. No user-facing surface + loses the text: `tui/data.py`'s `decision_pending` reads the RAW journal on the + operator's own machine, not this dump. + + The one-token case is the load-bearing one — grade the multi-word case alone and + the row stays green with the routing deleted, because the fallback happens to + redact it. + + Ablation: drop `question` from `_JOURNAL_DROP_FIELDS` and the one-token absence + assertion reddens (the multi-word one does not — which is the point).""" + pseudo = sanitize.Pseudonymizer(salt=b"fixed") + one_token = diagnostics._scrub_entry( + {"ts": 2.0, "kind": "decision-pending", "dw_id": "DW-7", "question": "AcmeVault"}, + pseudo, + {}, + 1.0, + ) + + assert "question" not in one_token + assert one_token["question_present"] is True + assert "AcmeVault" not in json.dumps(one_token), "LEAK: one-token decision question" + # the dw id beside it is untouched — it is the record's correlation handle + assert one_token["dw_id"] == "DW-7" + + # an unset question still reports as absent rather than as set + empty = diagnostics._scrub_entry( + {"ts": 2.0, "kind": "decision-pending", "dw_id": "DW-7", "question": ""}, + pseudo, + {}, + 1.0, + ) + assert empty["question_present"] is False + + rendered = json.dumps([one_token, empty]) + for canary in CANARIES: + assert canary not in rendered, f"LEAK: {canary!r}" + + def test_structure_is_preserved(project): run_dir = _seed_run(project.project) diag, _pseudo, _combined = _render_all([run_dir]) @@ -1749,14 +2064,22 @@ def test_diag_surfaces_the_split_code_root_and_the_task_generation(project): `paused_reason_present` / `worktree_isolated` style, and a small counter. The path itself must NOT appear — that is what `_JOURNAL_DROP_FIELDS` drops. - Ablation: delete `repo_root_diverges=` from `collect_run` (or `generation=` from - `_task_diag`) and this reddens on the corresponding assertion; deleting the field - from the dataclass reddens as a TypeError at construction. + `escalations_resolved_upto` (DW-11) is projected on the same warrant and asserted + here for the same reason: a task whose older escalations are filtered out of + `context.json` dumps identically to one that only ever raised the entries shown, + so a support bundle cannot explain a short resolve context without it. A counter + too — it indexes `task.sessions`, so it carries no customer content. + + Ablation: delete `repo_root_diverges=` from `collect_run` (or `generation=` / + `escalations_resolved_upto=` from `_task_diag`) and this reddens on the + corresponding assertion; deleting the field from the dataclass reddens as a + TypeError at construction. """ run_dir = _seed_run(project.project) state = load_state(run_dir) state.repo_root = str(project.project / "code-tree") state.tasks[STORY_KEY].generation = 2 + state.tasks[STORY_KEY].escalations_resolved_upto = 3 save_state(run_dir, state) diag, _pseudo, combined = _render_all([run_dir]) @@ -1764,6 +2087,7 @@ def test_diag_surfaces_the_split_code_root_and_the_task_generation(project): assert run.repo_root_diverges is True assert run.tasks[0].generation == 2 + assert run.tasks[0].escalations_resolved_upto == 3 # a presence flag, never the path — the same rule `repo` is dropped under assert "code-tree" not in combined @@ -1780,6 +2104,7 @@ def test_diag_repo_root_diverges_is_false_for_the_ordinary_layout(project): assert run.repo_root_diverges is False assert run.tasks[0].generation == 0 + assert run.tasks[0].escalations_resolved_upto == 0 def _md_task_row(md: str) -> list[str]: @@ -1800,19 +2125,28 @@ def test_the_markdown_report_carries_the_split_root_and_the_generation(project): `generation` rides beside `attempt` because that is the column pair a #705-class replay turns on: a collided re-drive and a healthy post-re-arm task agree on every - other cell in this row. + other cell in this row. DW-11's `escalations_resolved_upto` rides beside it on the + same warrant, stated verbatim in its own field comment: it is the only field that + separates "this story raised one escalation" from "its earlier ones are filtered + out of `context.json` as already answered", and that question is asked of a bug + report. Seeded to a value that is neither the attempt, the generation nor the + review cycle, so a cell reading a NEIGHBOUR cannot pass. Ablation: drop the `code root differs from project` line from `render_markdown` and both this test and the sibling below redden on their first assertion. Drop `{t.generation}` from the row f-string together with its header and separator cells - and this test reddens at `names[4]` (`"rev" != "gen"`) while the sibling reddens at - the row cell — as `"1" != "0"`, the review cycle shifted left rather than a missing - key, which is why the cell is read positionally and the three widths are compared. + and this test reddens at `names[4]` (`"esc-upto" != "gen"`) while the sibling + reddens at the row cell — the review cycle shifted left rather than a missing key, + which is why the cell is read positionally and the three widths are compared. Drop + `{t.escalations_resolved_upto}` the same way and this test reddens at `names[5]` + (`"rev" != "esc-upto"`); drop ONLY the row cell and it reddens on the width + comparison, which is what a skewed table actually looks like. """ run_dir = _seed_run(project.project) state = load_state(run_dir) state.repo_root = str(project.project / "code-tree") state.tasks[STORY_KEY].generation = 2 + state.tasks[STORY_KEY].escalations_resolved_upto = 3 save_state(run_dir, state) pseudo = sanitize.Pseudonymizer() @@ -1825,11 +2159,13 @@ def test_the_markdown_report_carries_the_split_root_and_the_generation(project): (rule,) = [ln for ln in md.splitlines() if ln.startswith("|---|")] names = [c.strip() for c in header.strip("|").split("|")] assert names[4] == "gen" + assert names[5] == "esc-upto" # header, separator and row must agree on width or the table renders skewed - assert len(cells) == len(names) == len(rule.strip("|").split("|")) == 12 + assert len(cells) == len(names) == len(rule.strip("|").split("|")) == 13 assert cells[3] == "2" # attempt, seeded by `_seed_run` assert cells[4] == "2" # generation — NOT the review cycle, which is 1 - assert cells[5] == "1" # review cycle, still in its own column + assert cells[5] == "3" # the DW-11 watermark, in its own column + assert cells[6] == "1" # review cycle, still in its own column # still a flag and a counter: the path itself never renders assert "code-tree" not in md diff --git a/tests/test_engine.py b/tests/test_engine.py index 0638596b..e0124973 100644 --- a/tests/test_engine.py +++ b/tests/test_engine.py @@ -5481,7 +5481,7 @@ def test_closes_deferred_lands_once_when_a_failed_commit_is_re_driven(project): # the resolve workflow's re-arm: a resolved re-drive, which is precisely the # recovery that PRESERVES the artifact folders' tracked content through # `safe_reset` — so a close left standing here would never be reverted. - rearm_escalation(engine.run_dir, isolated_redrive=False) + rearm_escalation(engine.run_dir, isolated_redrive=False, resolution_recorded=True) resumed, _ = resume_engine( project, @@ -9138,7 +9138,9 @@ def test_resolved_escalation_resume_skips_clean_rollback(project): assert summary.paused and summary.escalated == 1 assert load_state(engine.run_dir).tasks["1-1-a"].phase == Phase.ESCALATED - rearm_escalation(engine.run_dir, isolated_redrive=False) # the resolve workflow's re-arm step + rearm_escalation( + engine.run_dir, isolated_redrive=False, resolution_recorded=True + ) # the resolve workflow's re-arm step resumed, _ = resume_engine( project, @@ -9185,7 +9187,9 @@ def escalate_dirty(spec): summary = engine.run() assert summary.paused and summary.escalated == 1 - rearm_escalation(engine.run_dir, isolated_redrive=False) # the resolve workflow's re-arm step + rearm_escalation( + engine.run_dir, isolated_redrive=False, resolution_recorded=True + ) # the resolve workflow's re-arm step resumed, _ = resume_engine( project, @@ -9273,7 +9277,7 @@ def escalate_bound_repair(session): corrected = sp.read_text().replace("test spec", "human corrected frozen intent") sp.write_text(corrected) head_before_rearm = rev_parse_head(repo) - rearm_escalation(engine.run_dir, isolated_redrive=False) + rearm_escalation(engine.run_dir, isolated_redrive=False, resolution_recorded=True) assert rev_parse_head(repo) == head_before_rearm # no correction commit at re-arm assert read_frontmatter(sp)["status"] == "ready-for-dev" @@ -9844,7 +9848,9 @@ def halt_blocked(spec): assert task.phase == Phase.ESCALATED assert task.spec_file and Path(task.spec_file).name == sp.name # recorded despite HALT - rearm_escalation(engine.run_dir, isolated_redrive=False) # the resolve workflow's re-arm step + rearm_escalation( + engine.run_dir, isolated_redrive=False, resolution_recorded=True + ) # the resolve workflow's re-arm step assert read_frontmatter(sp)["status"] == "ready-for-dev" # re-drive will not HALT @@ -10080,7 +10086,7 @@ def test_intent_gap_restore_redrive_applies_patch_and_lands_done(project): assert engine.run().escalated == 1 rearm_escalation( - engine.run_dir, restore_patch=str(patch), isolated_redrive=False + engine.run_dir, restore_patch=str(patch), isolated_redrive=False, resolution_recorded=True ) # human confirmed the reading sp = spec_path(project, "1-1-a") assert read_frontmatter(sp)["status"] == "in-review" # routes step-01 -> step-04 @@ -10109,7 +10115,9 @@ def test_restore_redrive_prompt_points_at_the_spec(project): engine, _ = make_engine(project, [_escalate_with_patch(project, "1-1-a", patch)]) assert engine.run().escalated == 1 - rearm_escalation(engine.run_dir, restore_patch=str(patch), isolated_redrive=False) + rearm_escalation( + engine.run_dir, restore_patch=str(patch), isolated_redrive=False, resolution_recorded=True + ) seen: list[str] = [] resumed, adapter = resume_engine( project, engine, [_restoring_dev_effect(project, "1-1-a", seen)] @@ -10130,7 +10138,9 @@ def test_intent_gap_restore_reapplies_after_mid_redrive_rollback(project): patch = project.implementation_artifacts / "attempt.patch" engine, _ = make_engine(project, [_escalate_with_patch(project, "1-1-a", patch)]) assert engine.run().escalated == 1 - rearm_escalation(engine.run_dir, restore_patch=str(patch), isolated_redrive=False) + rearm_escalation( + engine.run_dir, restore_patch=str(patch), isolated_redrive=False, resolution_recorded=True + ) seen: list[str] = [] resumed, _ = resume_engine( @@ -10165,7 +10175,9 @@ def test_intent_gap_restore_escalates_when_resolution_commits_overlap(project): (repo / "src.txt").write_text("corrected by resolution\n") git(repo, "add", "src.txt") git(repo, "commit", "-q", "-m", "resolution: overlapping fix") - rearm_escalation(engine.run_dir, restore_patch=str(patch), isolated_redrive=False) + rearm_escalation( + engine.run_dir, restore_patch=str(patch), isolated_redrive=False, resolution_recorded=True + ) seen: list[str] = [] resumed, _ = resume_engine(project, engine, [_restoring_dev_effect(project, "1-1-a", seen)]) @@ -10552,7 +10564,9 @@ def test_resume_re_gates_a_human_armed_re_drive(project): ) engine, _ = make_engine(project, [escalating]) assert engine.run().escalated == 1 - rearm_escalation(engine.run_dir, isolated_redrive=False) # the resolve workflow's re-arm step + rearm_escalation( + engine.run_dir, isolated_redrive=False, resolution_recorded=True + ) # the resolve workflow's re-arm step assert load_state(engine.run_dir).tasks["1-1-a"].attempt == 0 # the confusable state # a gate lands on the story while the operator is resolving it write_gated_ledger(project, {"DW-1": ("open", ["gate: 1-1"])}) @@ -11079,7 +11093,7 @@ def test_session_env_fault_pauses_dev_without_burning_budget(project): assert end["env_fault_evidence"] == evidence # the resolve workflow's re-arm step restores the attempt budget - rearm_escalation(engine.run_dir, isolated_redrive=False) + rearm_escalation(engine.run_dir, isolated_redrive=False, resolution_recorded=True) assert load_state(engine.run_dir).tasks["1-1-a"].attempt == 0 @@ -12267,7 +12281,9 @@ def test_resume_with_epic_filter_stays_in_scoped_epic(project): assert summary.paused and summary.escalated == 1 assert engine.state.current_epic == 9 - rearm_escalation(engine.run_dir, isolated_redrive=False) # the resolve workflow's re-arm step + rearm_escalation( + engine.run_dir, isolated_redrive=False, resolution_recorded=True + ) # the resolve workflow's re-arm step resumed, _ = resume_engine( project, engine, @@ -12329,7 +12345,9 @@ def test_resolved_redrive_reescalates_instead_of_deferring(project): summary = engine.run() assert summary.paused and summary.escalated == 1 - rearm_escalation(engine.run_dir, isolated_redrive=False) # human resolved; re-drive re-armed + rearm_escalation( + engine.run_dir, isolated_redrive=False, resolution_recorded=True + ) # human resolved; re-drive re-armed # re-drive never reaches `done` (env still blocked): both attempts land at # in-progress with no escalation — the exact non-convergence that used to defer resumed, _ = resume_engine( @@ -16787,3 +16805,79 @@ def test_notice_reason_bound_is_an_upper_bound_not_an_equality(): short = _notice_reason("short first line\nthe evidence lives here") assert short == "short first line […]" # marked well under the cap assert len(short) < NOTICE_REASON_MAX + + +def test_llm_authored_preference_keys_cannot_hijack_journal_reserved_names(project): + """`_review_and_commit` splats a review session's own `result.json` escalation + entries into `journal.append`, so an LLM chooses the journal FIELD NAMES. Some + collide with the bound call, while others can replace metadata `append` owns. + + Reproduced before the fix: an entry carrying `kind` raised + `TypeError: Journal.append() got multiple values for argument 'kind'`, and + `story_key` the same, so a single invented key ABORTED THE WHOLE REVIEW LEG. + `self` collides with the bound-method receiver. `ts` did not raise and was worse + for it — `append` builds + `{"ts": now, "kind": kind, **fields}`, so a supplied `ts: 0` silently replaced + the real clock and every relative offset a diagnostic dump computes off it. + `log_task` and `log_pos` also did not raise: `append` stamps them with + `setdefault`, so caller values silently won, forged the pane-log pointer, and let + an identifier-shaped `log_pos` survive the diagnostic scrubber verbatim. + + Driven through a real run rather than by calling the filter directly: the + defect was the CALL SITE forwarding unfiltered keys, and a unit test over + `_JOURNAL_RESERVED_KEYS` would pass with the site left unpatched. + + Ablation: drop the `_JOURNAL_RESERVED_KEYS` comprehension in + `_review_and_commit` and this reddens with the TypeError above.""" + + def hostile_review_effect(spec): + sp = spec_path(project, "1-1-a") + baseline = _spec_baseline(sp) + write_spec(sp, "done", baseline) + set_sprint(project, "1-1-a", "done") + return SessionResult( + status="completed", + result_json={ + "workflow": "auto-dev", + "story_key": "1-1-a", + "spec_file": str(sp), + "baseline_commit": baseline, + "status": "done", + "followup_review_recommended": False, + "escalations": [ + { + "type": "preference", + "severity": "PREFERENCE", + "detail": "prose", + # journal-owned names, all LLM-authored here + "self": "hijacked-receiver", + "kind": "hijacked-kind", + "story_key": "9-9-not-this-story", + "ts": 0, + "log_task": "9-9-forged-story", + "log_pos": "AcmeVault", + } + ], + }, + ) + + write_sprint(project, {"epic-1": "backlog", "1-1-a": "ready-for-dev"}) + engine, _ = make_engine(project, [dev_effect(project, "1-1-a"), hostile_review_effect]) + + summary = engine.run() # the TypeError made this raise + + assert summary.done == 1 + entries = [ + json.loads(ln) + for ln in (engine.run_dir / "journal.jsonl").read_text(encoding="utf-8").splitlines() + ] + (pref,) = [e for e in entries if e["kind"] == "preference-escalation"] + # the journal's own names survived, none of them the LLM's + assert pref["kind"] == "preference-escalation" + assert pref["story_key"] == "1-1-a" + assert pref["ts"] > 1_000_000_000, "an LLM-supplied ts replaced the real clock" + assert pref["log_task"] != "9-9-forged-story" + assert isinstance(pref["log_pos"], int) + assert "AcmeVault" not in json.dumps(pref), "an LLM-supplied log_pos reached the journal" + # ...and the declared schema still rode through untouched + assert pref["type"] == "preference" and pref["detail"] == "prose" diff --git a/tests/test_engine_worktree.py b/tests/test_engine_worktree.py index 3a3bf153..ec0d9412 100644 --- a/tests/test_engine_worktree.py +++ b/tests/test_engine_worktree.py @@ -2101,7 +2101,12 @@ def commit_fails(*_a, **_k): assert not project.deferred_work.exists() # the row is only in the doomed worktree monkeypatch.setattr(verify, "finalize_commit", real_finalize) - assert runs.rearm_escalation(engine.run_dir, "1-1-a", isolated_redrive=True) == "1-1-a" + assert ( + runs.rearm_escalation( + engine.run_dir, "1-1-a", isolated_redrive=True, resolution_recorded=True + ) + == "1-1-a" + ) state = load_state(engine.run_dir) state.clear_pause() @@ -5762,7 +5767,12 @@ def commit_fails(*_a, **_k): assert _ledger_entry(project, "DW-1").open monkeypatch.setattr(verify, "finalize_commit", real_finalize) - assert runs.rearm_escalation(engine.run_dir, "1-1-a", isolated_redrive=True) == "1-1-a" + assert ( + runs.rearm_escalation( + engine.run_dir, "1-1-a", isolated_redrive=True, resolution_recorded=True + ) + == "1-1-a" + ) state = load_state(engine.run_dir) state.clear_pause() diff --git a/tests/test_generic_tmux.py b/tests/test_generic_tmux.py index ea8e4800..d7737e1c 100644 --- a/tests/test_generic_tmux.py +++ b/tests/test_generic_tmux.py @@ -29,6 +29,7 @@ from bmad_loop.adapters.multiplexer import MultiplexerError from bmad_loop.adapters.profile import get_profile from bmad_loop.bmadconfig import ProjectPaths +from bmad_loop.journal import TASK_CYCLE_ARTIFACTS from bmad_loop.model import TokenUsage from bmad_loop.policy import LimitsPolicy, NotifyPolicy, Policy from bmad_loop.signals import HookEvent @@ -3137,30 +3138,39 @@ def test_start_session_resets_reused_task_log(tmp_path): assert _classify(adapter, "timeout", task_id=task_id).env_fault is False -def test_start_session_drops_a_reused_task_dirs_escalation(tmp_path): - """The sweep skill writes `escalation.json` into tasks// and - `resolve._gather_escalations` reads it beside result.json. A re-armed run reuses - task_ids, so a prior cycle's escalation left there is handed to whatever session - lands on the id next — the same reuse hazard result.json's unlink already covers, - against a third reader. An ABSENT file must still start cleanly (missing_ok).""" +def test_start_session_drops_every_reused_task_cycle_artifact(tmp_path): + """A re-armed run reuses task_ids, so anything a prior cycle left in + tasks// is handed to whatever session lands on the id next — and + `resolve._gather_escalations` reads those files back to decide what to show the + operator. An ABSENT file must still start cleanly (missing_ok). + + Iterates `journal.TASK_CYCLE_ARTIFACTS` rather than naming the files, so this + covers the list rather than today's two entries: a third artifact added to the + constant is asserted here with no edit. The parity with + `OpencodeHTTPAdapter.start_session` used to be a claim in a docstring — both + adapters now loop over the same constant, and + `test_portability_guard.test_task_cycle_artifacts_named_only_through_the_constant` + refuses a bare literal that would let one drift from the other.""" mux = _StartSessionMux() adapter = make_adapter(tmp_path, mux=mux) adapter._ensure_session = lambda cwd: None # skip the tmux server plumbing task_id = _ENV_FAULT_TASK task_dir = adapter.tasks_dir / task_id task_dir.mkdir(parents=True, exist_ok=True) - stale = task_dir / "escalation.json" - stale.write_text( - json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "last cycle"}]}), - encoding="utf-8", - ) + assert TASK_CYCLE_ARTIFACTS, "the constant is the list under test; an empty one is vacuous" + stale = [task_dir / name for name in TASK_CYCLE_ARTIFACTS] + for path in stale: + path.write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "last cycle"}]}), + encoding="utf-8", + ) adapter.start_session(make_spec(tmp_path, task_id=task_id)) - assert not stale.exists() + assert [p for p in stale if p.exists()] == [] - # ...and with the file already gone the unlink is a no-op, not an error. What this - # second call asserts is that it RETURNS (the missing_ok path); re-asserting the - # file's absence would only restate the line above, since nothing re-created it. + # ...and with the files already gone the unlinks are no-ops, not errors. What this + # second call asserts is that it RETURNS (the missing_ok path); re-asserting their + # absence would only restate the line above, since nothing re-created them. assert adapter.start_session(make_spec(tmp_path, task_id=task_id)) is not None diff --git a/tests/test_model.py b/tests/test_model.py index 1b264bd6..50fd5ac0 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -291,6 +291,22 @@ def test_generation_defaults_zero_for_legacy_state(): assert StoryTask.from_dict(doc).generation == 0 +def test_escalations_resolved_upto_round_trips(): + task = StoryTask(story_key="1-1-a", epic=1, escalations_resolved_upto=3) + assert StoryTask.from_dict(task.to_dict()).escalations_resolved_upto == 3 + + +def test_escalations_resolved_upto_defaults_zero_for_legacy_state(): + """A `state.json` written before DW-11 must resume UNFILTERED. 0 is the value + `resolve._gather_escalations` reads as "nothing answered yet", so every escalation + the run recorded is still shown and nothing is reported withheld — byte-for-byte + today's behavior. Any other default would hide entries the human never saw, on a + run that was mid-escalation across the upgrade.""" + doc = StoryTask(story_key="1-1-a", epic=1).to_dict() + del doc["escalations_resolved_upto"] # state.json from before the field existed + assert StoryTask.from_dict(doc).escalations_resolved_upto == 0 + + def test_resolved_redrive_round_trips(): task = StoryTask(story_key="1-1-a", epic=1, resolved_redrive=True) assert StoryTask.from_dict(task.to_dict()).resolved_redrive is True diff --git a/tests/test_opencode_http.py b/tests/test_opencode_http.py index d2d61d9c..0088f48a 100644 --- a/tests/test_opencode_http.py +++ b/tests/test_opencode_http.py @@ -44,6 +44,7 @@ ) from bmad_loop.adapters.profile import get_profile from bmad_loop.bmadconfig import ProjectPaths +from bmad_loop.journal import TASK_CYCLE_ARTIFACTS from bmad_loop.model import TokenUsage from bmad_loop.policy import LimitsPolicy, NotifyPolicy, Policy from bmad_loop.process_host import ProcessHostError, get_process_host @@ -1290,28 +1291,35 @@ def test_missing_binary_is_a_clean_error(tmp_path): adapter.start_session(spec) -def test_start_session_drops_a_reused_task_dirs_escalation(tmp_path): +def test_start_session_drops_every_reused_task_cycle_artifact(tmp_path): """Parity with GenericAdapter: both adapters own a tasks// dir, so both must - drop a prior cycle's `escalation.json` — the file the sweep skill writes and - `resolve._gather_escalations` reads beside result.json — before a re-armed run - reusing the id lands there. No fake server needed: the unlink runs BEFORE - _spawn_server's PATH check raises, so a missing binary still exercises it.""" + drop a prior cycle's artifacts before a re-armed run reusing the id lands there. + No fake server needed: the unlinks run BEFORE _spawn_server's PATH check raises, + so a missing binary still exercises them. + + That parity is now STRUCTURAL rather than asserted twice in prose: both adapters + loop over `journal.TASK_CYCLE_ARTIFACTS`, this test iterates the same constant, + and `test_portability_guard.test_task_cycle_artifacts_named_only_through_the_constant` + refuses the bare literal that would let one adapter drift from the other. A third + artifact added to the constant is covered here with no edit.""" adapter = make_adapter(tmp_path, binary="definitely-not-a-real-binary-xyz") spec = SessionSpec(task_id="t-1", role="triage", prompt="p", cwd=tmp_path) task_dir = adapter.tasks_dir / "t-1" task_dir.mkdir(parents=True, exist_ok=True) - stale = task_dir / "escalation.json" - stale.write_text( - json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "last cycle"}]}), - encoding="utf-8", - ) + assert TASK_CYCLE_ARTIFACTS, "the constant is the list under test; an empty one is vacuous" + stale = [task_dir / name for name in TASK_CYCLE_ARTIFACTS] + for path in stale: + path.write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "last cycle"}]}), + encoding="utf-8", + ) with pytest.raises(OpencodeServerError, match="not found on PATH"): adapter.start_session(spec) - assert not stale.exists() + assert [p for p in stale if p.exists()] == [] - # ...and the ordinary case — no prior escalation — reaches the same spawn error, - # i.e. the unlink is missing_ok and did not become the failure itself + # ...and the ordinary case — nothing left behind — reaches the same spawn error, + # i.e. the unlinks are missing_ok and did not become the failure themselves with pytest.raises(OpencodeServerError, match="not found on PATH"): adapter.start_session(spec) diff --git a/tests/test_portability_guard.py b/tests/test_portability_guard.py index 36d3ae8d..06a91b47 100644 --- a/tests/test_portability_guard.py +++ b/tests/test_portability_guard.py @@ -7,13 +7,29 @@ in an allowlisted file and — outside the wholesale tmux quarantine — carries a ``# portability:`` ack on its line, so exceptions stay deliberate. -The same single-pass scan also carries the two non-POSIX quarantines that have the +The same single-pass scan also carries the non-POSIX quarantines that have the identical shape: AGENTS.md's "New core env vars register in ``envvars.py``; plugin-owned env-var families stay with their plugin" — see ``test_bmad_loop_env_reads_only_in_the_registry`` — and its "all git subprocess calls go through the ``_run_git`` chokepoint in ``verify.py``" — see ``test_no_git_invocation_outside_verify``. +Three later invariants ride the same machinery, each one previously held by +docstring prose alone: + +* the task-directory artifact names are ``journal.TASK_CYCLE_ARTIFACTS`` and not a + literal repeated per reader/writer — ``test_task_cycle_artifacts_named_only_through_the_constant`` +* a session task id is composed only in ``engine._session_task_id`` — + ``test_session_task_id_composed_only_at_the_chokepoint`` +* every journal field name a call spells is either routed by ``diagnostics``' + redaction tables — by name, or by name-and-kind — or declared benign: + ``test_journal_fields_are_routed_or_declared_benign``, with + ``test_journal_kinds_are_literal_or_the_position_is_declared`` holding the kind + half readable and ``test_journal_append_writes_only_accounted_fields`` covering + the two names ``Journal.append`` mints itself, which no call site spells. +* ``runs.rearm_escalation`` is called from exactly two places, each of which consults + liveness first — ``test_rearm_escalation_called_only_behind_a_liveness_gate``. + If this test flags something unexpected, fix the source (route it through the seam / a platform helper) rather than widening an allowlist. """ @@ -21,12 +37,20 @@ from __future__ import annotations import ast +import json +from collections import Counter from pathlib import Path import pytest import bmad_loop -from bmad_loop import envvars +from bmad_loop import diagnostics, envvars +from bmad_loop.journal import ( + JOURNAL_FILE, + SELF_MINTED_FIELDS, + TASK_CYCLE_ARTIFACTS, + Journal, +) SRC = Path(bmad_loop.__file__).resolve().parent # Marker an allowlisted exception line must carry. Written as ``# portability: …``; @@ -119,6 +143,460 @@ SPEC_ANCHOR_CHOKEPOINT = {"runs.py", "engine.py", "verify.py", "recovery_flow.py"} SPEC_PATH_FIELDS = {"spec_file", "dispatched_spec_file"} +# ``(file, name)`` of the ONE assignment that may spell the task-directory artifact +# names as literals: ``journal.TASK_CYCLE_ARTIFACTS`` itself. Constants inside that +# assignment's value are the definition, not a copy, so the scan skips them — the +# position idiom the git and verify exemptions use, rather than an allowlist entry +# that would also wave through a bare literal anywhere else in journal.py. +# +# Paired with the FILE on purpose: the same tuple re-declared in another module is a +# second copy, which is exactly what the guard exists to refuse. +TASK_ARTIFACT_DEFINITION = ("journal.py", "TASK_CYCLE_ARTIFACTS") + +# ``rel -> enclosing function -> the artifact names it may still spell as a bare +# literal``. Keyed by FUNCTION as well as by file — ``VERIFY_CLASSIFY_CHOKEPOINT``'s +# idiom — because the sanction is a POSITION: a second bare `"result.json"` grown +# anywhere else in `adapters/generic.py` would inherit a file-keyed exemption on its +# path alone, which is both the drift the guard exists to catch and the thing this +# comment used to claim was already impossible. +# +# Scoped by NAME inside that, for `ENV_READ_ALLOW`'s reason: being the sanctioned +# position buys `_result_path` the one name it declares and nothing wider. +# +# `adapters/generic.py::_result_path` is the one sanctioned single-name read: it +# answers "where does THIS task's result.json live", a genuinely single-artifact +# question that folding into the loop would not express. It carries no claim about +# `escalation.json`, so that name stays refused inside it. +TASK_ARTIFACT_LITERAL_ALLOW = { + "adapters/generic.py": {"_result_path": frozenset({"result.json"})}, +} + +# The one file allowed to COMPOSE a session task id, and within it only inside +# ``_session_task_id`` — keyed file -> the ONE enclosing function, like +# ``VERIFY_CLASSIFY_CHOKEPOINT``. Every mint site (`engine.py` ×3, `resolve.py`) +# calls it; none spells the format itself. +# +# The sanction is a POSITION, not the file: engine.py is where a fifth mint would +# most naturally be written (it already binds `task_id` three times), so a file-wide +# exemption would leave the invariant unguarded exactly where it matters. The +# function's own docstring states why every caller must be byte-identical — +# ``_resumable_session``'s resume match, and the ``-g`` re-arm discriminator that +# a hand-rolled fourth mint would omit (#705). +SESSION_TASK_ID_CHOKEPOINT = {"engine.py": "_session_task_id"} + +# The complete set of ``runs.rearm_escalation`` call sites, as +# ``(file, enclosing function)``. The re-arm transaction's own commit probe +# (``runs._rearm_commit_landed``) proves "did MY save_state land?" with nothing but +# ``(generation, phase)`` over the reloaded task, and that is a sufficient IDENTITY +# only under a sole-writer model: no engine advancing the task underneath, and one +# control command at a time. Its docstring argues that model from this enumeration. +# +# Prose cannot hold it. A third call site — or either existing gate deleted — leaves +# every test in the repo green while the probe's premise quietly becomes false, and +# the failure it opens is DW-79/DW-83's own shape: a spec left re-armed against a task +# the run still calls ESCALATED. So the enumeration is scanned instead of asserted. +# +# Deliberately NOT a lock and not a durable per-re-arm token: the spec's ``Never`` +# forbids both (a lock only ``rearm_escalation`` takes excludes nobody; a token buys a +# precision ``save_state`` cannot honour). It forbids no guard, and this is the cheap +# half — it does not make overlapping callers safe, it makes the day someone adds one +# impossible to miss. Overlapping control commands stay out of the model, as DW-93. +REARM_ESCALATION_CALLERS = { + ("cli.py", "cmd_resolve"), + ("tui/app.py", "_do_rearm"), +} + +# What counts as consulting liveness, matched as a substring of the callee's name +# because the two sites legitimately spell it differently and neither spelling is more +# correct: the CLI calls ``runs.engine_liveness`` directly, the TUI goes through +# ``self._resolve_blocked_by_liveness`` (which reaches ``runs.liveness``, the pid-file +# sibling sharing ``probe_liveness``). Pinning either exact name would redden on a +# rename that changes nothing, while the substring still reddens on the deletion this +# guard exists for. +# +# What the gate establishes is that the engine is not PROVABLY alive, not that it is +# proven dead — ``"unknown"`` proceeds under ``--force`` in ``cmd_resolve``, and the +# TUI counts it as blocking only for a pid-backed run. This guard therefore grades +# that the result controls a terminating branch before the call; the caller-level +# tests pin the exact alive/unknown policy on the two real surfaces. +LIVENESS_GATE_MARK = "liveness" + +# The journal field names ``diagnostics`` routes BY NAME, read off the live module +# rather than copied, so the guard cannot drift from the tables it grades: add a row +# there and the corresponding producer stops being an offender with no edit here. +# Three tables, because these three are the by-name routing decisions — an alias, a +# drop, or a key-list reduction. Anything else falls through to +# ``sanitize.scrub_json``, which fails closed only by accident of a value's shape. +# +# ``_JOURNAL_KIND_ALIAS_FIELDS`` is deliberately NOT flattened in here. It routes by +# ``(kind, name)``, and folding it into a by-name union says ``target`` is routed +# everywhere — including on the ``board-advance-*`` family, where that module's own +# comment says by-name routing would be WRONG. Flattened, the guard read +# ``journal.append("unit-merge-failed", target=branch)`` — a NEW kind reusing the +# name — as routed, while ``_scrub_entry`` handed it to ``scrub_json`` and shipped +# the branch verbatim. See ``JOURNAL_KIND_ROUTED_FIELDS`` for the scoped form. +JOURNAL_ROUTED_FIELDS = ( + frozenset(diagnostics._JOURNAL_ALIAS_FIELDS) + | diagnostics._JOURNAL_DROP_FIELDS + | diagnostics._JOURNAL_KEYLIST_FIELDS +) + +# ``kind -> the field names routed on THAT kind only``, read off the same module so +# the guard still cannot drift from it. A name here is routed on its own kinds and +# unrouted everywhere else, which is the distinction the flattened union destroyed. +JOURNAL_KIND_ROUTED_FIELDS = { + kind: frozenset(row) for kind, row in diagnostics._JOURNAL_KIND_ALIAS_FIELDS.items() +} + +# ``kind -> field names declared benign on that kind alone`` — the kind-scoped twin of +# ``JOURNAL_BENIGN_FIELDS``, and it exists for the same field the routing table does. +# ``engine``'s board-advance carry paths journal ``target`` carrying a sprint STATUS +# ("done"), not a branch; ``diagnostics``' ``_JOURNAL_KIND_ALIAS_FIELDS`` comment is +# explicit that aliasing those would destroy the field a maintainer reads the record +# for. Declared per kind rather than by adding ``target`` to the by-name benign set, +# which would also wave through a branch-carrying ``target`` on a kind nobody has +# looked at — exactly the hole the flattening left. +JOURNAL_KIND_BENIGN_FIELDS = { + "board-advance-carried": frozenset({"target"}), + "board-advance-carry-failed": frozenset({"target"}), + "board-advance-carry-foreign-dirt": frozenset({"target"}), + "board-advance-carry-uncommitted": frozenset({"target"}), +} + +# Every OTHER field name journalled today: a declared inventory, not a per-name +# audit. Nobody has argued each of these is safe unrouted; what the list records is +# that they are the set that existed when the guard landed. That is the whole claim, +# and it is worth making — field name #132 cannot appear without someone deciding +# whether it needs routing, which is the decision DW-82 measured nothing forcing. +# +# ⚠️ Adding a name here is that decision, made in the "no routing needed" direction. +# Make it deliberately: a name carrying a story key, a branch, a sha, a spec +# filename, a path, or free text belongs in a `diagnostics` table instead. Adding a +# routing row there for a field that does not need one is equally wrong — it would +# pseudonymize a value a maintainer reads the record for (see +# `_JOURNAL_KIND_ALIAS_FIELDS`' `target` for that failure in the other direction). +# +# ⚠️ STATED BOUND, so nobody reads more into this than it says: the guard catches a +# rename OUT of the tables into unclaimed space — the measured `patch` → `patch_path` +# ablation. It does NOT catch a rename INTO a name one of these sets already holds. +# Respell `recovery_flow.py`'s `patch=` as `path=`, `ref=` or `name=` and every +# assertion here stays green while the value stops being dropped, because the guard +# grades the NAME against a set and all three of those names are in it. Only +# `tests/test_diagnostics.py` can see that, and only if it has a row for the record. +JOURNAL_BENIGN_FIELDS = frozenset( + { + "action", + "actions", + "adapter", + "adapter_dev", + "adapter_review", + "already_resolved", + "attempt", + "blocked", + "blocking", + "budget", + "budget_mode", + "budget_weighted", + "bundles", + "bundles_not_run", + "cache_read_weight", + "cache_read_weight_was", + "cap", + "checkout_dirty", + "checkpoint", + "code_root_changed", + "command_index", + "commits", + "condition", + "contradiction", + "converted", + "count", + "cycle", + "cycles", + "decision", + "decisions", + "deduped", + "dropped", + "dw_id", + "effect", + "entries", + "entries_now", + "env_fault", + "env_fault_evidence", + "epic", + "errors", + "expired_clock", + "failed", + "field", + "files", + "finished", + "fired_at", + "flat_remainder", + "followup_damped", + "followup_review_recommended", + "frm", + "graceful", + "harvest_attempt", + "head", + "id_collisions", + "items", + "kept", + "key", + "ledger", + "log_pos", + "malformed", + "mode", + "model", + "name", + "next", + "normalized", + "ok", + # `old_baseline` is NOT here any more: it moved to `_JOURNAL_ALIAS_FIELDS` + # (the `commit` namespace) once a second producer — + # `rearm-commits-probe-failed` — forced the decision this set's own warning + # describes, and on the same footing as the `question` note above: it was a + # live leak, just an intermittent one. Unrouted, a real 40-hex sha usually + # collapses to `` at `_scrub_str`'s secret check — but only + # usually. Real shas straddle that bar, and about one in twenty-five sampled + # from this repo's own history ships VERBATIM. Routing also restores the + # correlation the alias table exists to preserve: even on the shas the + # fallback does catch, `` left the two records naming one + # baseline unable to be seen as naming the same one. Left as a note rather + # than a silent deletion, because a name leaving this set is the guard working + # — a benign declaration that turned out to be wrong. + "open", + "open_now", + "original", + "owed_after_implement", + "path", + "paths", + "phase", + "platform", + "plugin", + "plugins", + "policy_changed", + "preserve_ref", + "problem", + # `question` is NOT here any more: it moved to `_JOURNAL_DROP_FIELDS` + # (schema v3) once a one-token `decision-pending` question was shown to + # ship verbatim. Left as a note rather than a silent deletion, because a + # name leaving this set is the guard working — a benign declaration that + # turned out to be wrong. + "rc", + "re_review_capped", + "rearmed", + "record", + "redrive", + "ref", + "refiled", + "refs", + "refused", + "remaining", + "reset_from", + "restore", + "returncode", + "role", + # The outcome of an aborted re-arm's spec rollback (`rearm-aborted`), one of + # FOUR literal enum strings the producer chooses (`restored`, `unchanged`, + # `unknown`, `failed`). Benign rather than routed: + # it names no customer artifact and IS the field both operator surfaces read + # the record for, so an alias would destroy it (the failure + # `_JOURNAL_KIND_ALIAS_FIELDS`' `target` row documents in the other direction). + "rollback", + "run_id", + "run_type", + "security_config_changed", + "sentinel", + "sentinel_kind", + "session_status", + "session_vanished", + "signum", + "site", + "skip", + "source", + "spec_folder", + "stage", + "state_kind", + "status", + "stderr_bytes", + "stderr_captured_bytes", + "stderr_truncated", + "stdout_bytes", + "stdout_captured_bytes", + "stdout_truncated", + "strategy", + "teardown_s", + "to", + "tokens", + "tokens_weighted", + "tolerated", + "total", + "trigger", + "verification_sequence", + "verification_stage", + "via", + "weighted", + "workflow", + "worktree", + "zero_diff", + } +) + +# Field names NO call site spells as a keyword, because ``Journal.append`` mints them +# itself: ``entry.setdefault("log_task", …)`` and ``entry.setdefault("log_pos", size)`` +# on every entry written while a pane log is active. ``log_task`` is routed (a story +# alias); ``log_pos`` is a byte offset and is declared benign above. +# +# The static scan reads CALL SITES, so it cannot see either of them — which means the +# sibling guard's "every field name a journal producer writes" claim is true only of +# the fields a call spells. ``test_journal_append_writes_only_accounted_fields`` +# closes that from the other side by RUNNING an append and reading the entry back; +# this set is what stops the staleness check below from calling ``log_pos`` dead. +# +# READ FROM ``journal``, not restated: ``diagnostics._scrub_entry`` exempts the same +# pair from the fail-closed arm it applies to a declared-schema kind, and a literal +# copy here would let this guard and that exemption drift apart silently — which is +# the failure mode DW-82 exists to remove, applied to the guard itself. +JOURNAL_SELF_MINTED_FIELDS = SELF_MINTED_FIELDS + +# ``(file, enclosing function) -> the field names that actually flow through it`` for +# every ``journal.append(**name)`` whose keys are NOT statically resolvable. An +# unresolved splat is a HOLE in the inventory above — the guard cannot tell whether a +# new field arrived through it — so it fails loud and each hole is declared here with +# why it is one, rather than being silently skipped. A new splat site anywhere else +# reddens the guard until someone either makes its keys resolvable or adds a line here. +# +# All four are unresolvable for the same structural reason: the dict is not built +# from literals in the calling function. The VALUES are an inventory read off the +# producer, not an assertion the scan can check — they are what keeps the staleness +# check on ``JOURNAL_BENIGN_FIELDS`` from calling a splat-borne name dead, and they +# are the honest answer to "which names does this hole let through". +JOURNAL_SPLAT_ALLOW = { + # `streams` keys are computed — `f"{kind}_path"` and its three siblings over a + # fixed (stdout, stderr) loop — so the resolver cannot read them and the argument + # for the hole is the POSITION. Said plainly because the previous comment argued + # by VALUE TYPE ("numbers and booleans") while the invariant it exempts is + # NAME-based: the two `*_path` names are routed (`_JOURNAL_DROP_FIELDS`); the + # other six are declared benign BY NAME, below. ⚠️ A NEW key added inside this + # `streams` dict is still invisible to the guard — that is what the hole IS, and + # no property of its value changes it. + ("engine.py", "_journal_verify_command_results"): frozenset( + { + "stdout_path", + "stderr_path", + "stdout_bytes", + "stderr_bytes", + "stdout_captured_bytes", + "stderr_captured_bytes", + "stdout_truncated", + "stderr_truncated", + } + ), + # `pref` comes from `preference_escalations(result_json)` — LLM-authored keys out + # of a session's own result.json. Not statically knowable in principle, not just + # in this scan, so the OFF-SCHEMA half of this hole can never be inventoried. + # + # The three names below are the half that can: they are the record's declared + # schema, and they are the names whose VALUES still reach the dump (everything + # else on this kind collapses to a presence marker). Asserted against + # `diagnostics._JOURNAL_KIND_SCHEMAS` by + # `test_journal_routing_tables_are_read_from_diagnostics`, so this inventory and + # that table cannot disagree. + # + # What covers it is `diagnostics._JOURNAL_KIND_SCHEMAS`, which declares + # `preference-escalation`'s record to be `{type, severity, detail}` and collapses + # every other key on that kind to `_present`. This comment used to say the + # REDACTION FALLBACK covered it, which was verified false: `scrub_json` is the + # IDENTITY on an identifier-shaped scalar, so `customer="AcmeVault"` came back + # byte-identical while this allowlist entry read as accounted for. A comment that + # names the wrong mechanism is how the next reader concludes a hole is closed + # when it is not. + # + # The hole this entry declares is therefore narrower than it looks, and it is + # still a hole: the key NAMES remain LLM-authored and still reach the dump as + # `_present` markers. That residual was weighed against a name-free + # `unrouted_field_count` collapse and DELIBERATELY ACCEPTED on 2026-08-30 — see + # `_JOURNAL_KIND_SCHEMAS`. It is decided, not outstanding. + ("engine.py", "_review_and_commit"): frozenset({"type", "severity", "detail"}), + # `self._session_end_extras(result)` is a method call, and that method builds its + # dict with `extras.update(...)` — unresolvable at the call site and at the + # definition. The names below are read off `engine._session_end_extras`, and five + # of them (`fired_at`, `teardown_s`, `expired_clock`, `budget_weighted`, + # `budget_mode`) have NO other producer anywhere: the previous comment's claim + # that these keys "are in the benign inventory because other sites journal them + # explicitly" was simply false. They are in it because THIS declaration puts them + # there. ⚠️ A new key added inside `_session_end_extras` is still invisible. + ("engine.py", "_run_session"): frozenset( + { + "fired_at", + "teardown_s", + "expired_clock", + "budget_weighted", + "budget", + "budget_mode", + "env_fault", + "env_fault_evidence", + "session_vanished", + } + ), + # The plugin bus's `_log` forwards its OWN `**fields` parameter, so the keys + # belong to each CALLER and there is no store in this function to resolve. The + # callers' keywords are read at their own sites — but ONLY because + # `JOURNAL_FORWARDERS` declares `_log` a journal write. Before that they were + # unreachable: `_is_journal_write` matched `.append(...)` alone, the four + # `self._log(...)` sites were never read, and `rc` and `blocking` sat in neither + # routing set with this guard green. That is what the old comment's "the scan + # reads them directly at their own sites" asserted and did not do. + ("plugins/bus.py", "_log"): frozenset(), +} + +# ``(file, function name)`` of every helper that FORWARDS to ``journal.append`` with a +# ``**kwargs`` of its own. A call to that NAME inside that FILE counts as a journal +# write, so the forwarder's callers put their explicit keywords into the inventory +# instead of stopping at a wall. +# +# The forwarder's own `self._journal.append(kind, **fields)` stays an unresolvable +# splat — its parameter has no store to resolve — so both this entry and the +# `JOURNAL_SPLAT_ALLOW` one are needed, and they say different things: this one makes +# the CALLERS visible, that one declares the forwarder's own hole. +JOURNAL_FORWARDERS = {("plugins/bus.py", "_log")} + +# ``(file, enclosing function)`` of every journal write whose KIND is not a string +# literal. Kind-scoped routing (`JOURNAL_KIND_ROUTED_FIELDS` / +# `JOURNAL_KIND_BENIGN_FIELDS`) cannot be evaluated at such a call, so — exactly like +# an unresolvable splat — the site fails loud rather than being graded against a kind +# the scan had to guess. +# +# Declaring a position waives the KIND resolution and NOTHING else: a kind-scoped +# name at one of these sites is still refused, because nothing here can prove which +# kind it lands on. +JOURNAL_DYNAMIC_KIND_ALLOW = { + # `kind` is a keyword parameter defaulting to `review-skipped`, flipped to + # `review-skipped-awaiting-operator` by the park path. Journals `story_key` only. + ("engine.py", "_skip_review_and_commit"), + # `kind` is chosen by the two ledger-close call sites. Journals `story_key` and + # `dw_ids` only. + ("sweep.py", "_close_bundle_ledger_when_spec_status"), + # Four writes, each an f-string over the `family` loop variable: + # `attempt-preserve` / `attempt-preserve-dirty` × `-pruned` / `-prune-failed`. + ("recovery_flow.py", "prune_preserve_refs"), + # The forwarder passes its caller's `kind` straight through; every CALLER spells + # a literal, and `JOURNAL_FORWARDERS` is what lets the scan read them there. + ("plugins/bus.py", "_log"), +} + +# The receivers a ``.append(...)`` call must hang off to be a journal write. Matched +# on the trailing name so `self.journal`, a bare `journal` parameter and +# `self._journal` (the plugin bus's optional handle) all resolve — the three +# spellings in the tree. +# +# ⚠️ STATED BOUND: a LOCALLY ALIASED handle is invisible. `j = self.journal` followed +# by `j.append(kind, customer_email=x)` produces no finding (verified by running it +# through `_scan_source`). No such site exists in the tree today, and resolving the +# binding would be `_call_aliases`' shape rather than a new idea — but the +# guard does not do it, and a reader must not assume it does. +JOURNAL_RECEIVERS = {"journal", "_journal"} + # Files that may name a bare POSIX path, each on a line carrying a `# portability:` # ack. process_host.py's Linux identity reader walks `/proc//stat` behind a # sys.platform branch; the Unity teardown scripts are POSIX-only. verify.py is the @@ -466,8 +944,8 @@ def _called_name(func: ast.expr) -> str | None: return None -def _verify_call_aliases(tree: ast.AST, target: str) -> frozenset[str]: - """Bare names statically bound to one guarded verify-call target. +def _call_aliases(tree: ast.AST, target: str) -> frozenset[str]: + """Bare names statically bound to one guarded call target. The call-site spelling alone misses the ordinary Python aliases a future caller may use: rename-on-import and a local assignment from either the @@ -534,6 +1012,289 @@ def _names_verify_classifier(func: ast.expr, aliases: frozenset[str] = frozenset return _names_guarded_verify_call(func, "verify_command_results_outcome", aliases) +def _is_str_composition(node: ast.expr) -> bool: + """Whether this expression BUILDS a string rather than naming one, in three + spellings — NOT "the three spellings a hand-minted task id can take", which is + an overclaim the shapes below cannot support. + + ``JoinedStr`` is the f-string. ``BinOp`` with a str ``Constant`` on either side + covers both concatenation (``story + "-review-1"``) and percent formatting + (``"%s-dev-%d" % (key, n)``), whose operator is also a ``BinOp``. The third is + ``"…".format(…)`` on a literal receiver. + + Three real compositions this deliberately does NOT recognise, verified silent: + ``"-".join([key, "dev", "1"])``, ``fmt % (key, n)`` where ``fmt`` is a Name bound + to the format string, and any of the three assembled a statement earlier and + forwarded through a variable. See the ``NOT COVERED`` note on the detector for + why the boundary sits where it does. + + A ``Name``, ``Attribute``, ``Subscript`` or ordinary ``Call`` is deliberately NOT + a composition: those FORWARD a string someone else made, which is what every + sanctioned mint site does with the chokepoint's return value.""" + if isinstance(node, ast.JoinedStr): + return True + if isinstance(node, ast.BinOp) and any( + isinstance(side, ast.Constant) and isinstance(side.value, str) + for side in (node.left, node.right) + ): + return True + return ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr == "format" + and isinstance(node.func.value, ast.Constant) + and isinstance(node.func.value.value, str) + ) + + +def _is_bare_str(node: ast.expr) -> bool: + return isinstance(node, ast.Constant) and isinstance(node.value, str) + + +def _mint_candidates(node: ast.expr, depth: int = 0): + """``(sub-expression, depth)`` for every value position that could be minting a + string here, where depth counts the CALL boundaries crossed to reach it. + + Conditionals and boolean fallbacks are descended at the same depth, since both + branches are the same value position (``task_id = f"…" if x else base``). + + Call arguments are descended too, in EVERY position, because a call is the shape + a mint hides behind in both of them. In a return it is the sanitizer the + chokepoint itself uses — ``return safe_segment(f"{story_key}-{part}-{seq}{gen}")`` + — and in a binding it is the same line copied into one: ``task_id = + safe_segment(f"{key}-dev-1")`` is the most likely fifth mint precisely because it + is the chokepoint's own body moved. Refusing to descend there left that shape + silent (verified), and it omits the ``-g`` suffix, which is #705 re-opened. + + Depth is what makes descending safe. A bare string Constant is a mint only at + depth 0 (``task_id = "triage-1"``); at depth it is an ARGUMENT and flagging it + would hit ``os.environ.get("BMAD_LOOP_TASK_ID")`` and the ``"dev"`` part in every + sanctioned ``_session_task_id(key, "dev", seq, gen)`` call. A COMPOSITION is a + mint at any depth: nothing legitimate hands a freshly built string to a call in a + ``task_id`` position.""" + yield node, depth + if isinstance(node, ast.IfExp): + yield from _mint_candidates(node.body, depth) + yield from _mint_candidates(node.orelse, depth) + elif isinstance(node, ast.BoolOp): + for value in node.values: + yield from _mint_candidates(value, depth) + elif isinstance(node, ast.Call): + for arg in [*node.args, *(kw.value for kw in node.keywords)]: + yield from _mint_candidates(arg, depth + 1) + + +def _is_journal_write(node: ast.AST, rel: str) -> bool: + """Whether this node writes a journal entry — a ``.append(...)`` call in + each of the three receiver spellings the tree uses (see ``JOURNAL_RECEIVERS``), + or a call to one of this file's declared ``JOURNAL_FORWARDERS``. + + The forwarder half is not a convenience. ``plugins/bus.py::_log`` takes its own + ``**fields`` and hands them to ``self._journal.append``, so its four call sites + spell keywords that reach the journal while matching nothing the ``.append`` + scan looks at — `rc` and `blocking` were in neither routing set with this guard + green. Keyed ``(file, name)``: a ``_log`` elsewhere forwards to something else. + + The receiver's qualifier is ignored for ``_called_name``'s reason: an aliased + MODULE handle reaches the same method. A locally aliased receiver is a stated + bound — see ``JOURNAL_RECEIVERS``.""" + if not isinstance(node, ast.Call): + return False + name = _called_name(node.func) + if name is None: + return False + if (rel, name) in JOURNAL_FORWARDERS: + return True + return ( + isinstance(node.func, ast.Attribute) + and name == "append" + and _called_name(node.func.value) in JOURNAL_RECEIVERS + ) + + +def _dict_literal_keys(value: ast.expr) -> set[str] | None: + """The string keys of a dict literal, or None when any key is not a static + string. ``{**other}`` yields a ``None`` key node and is unresolvable by + definition; a conditional between two literals resolves to their union, which is + how ``engine._run_inner`` builds its ``extras``.""" + if isinstance(value, ast.Dict): + keys: set[str] = set() + for key in value.keys: + if not (isinstance(key, ast.Constant) and isinstance(key.value, str)): + return None + keys.add(key.value) + return keys + if isinstance(value, ast.IfExp): + body, orelse = _dict_literal_keys(value.body), _dict_literal_keys(value.orelse) + return None if body is None or orelse is None else body | orelse + return None + + +def _journal_splat_keys(fn: ast.AST | None, name: str) -> set[str] | None: + """The keys a ``**name`` splat can carry, resolved through the same-function + literal stores that build it, or None when ANY store is not statically + resolvable. + + Fails closed on purpose, in four directions, because a partially-resolved + splat would under-report and read as green: an augmented assignment + (``fields += …``), a method mutation (``fields.update(…)``, + ``fields.setdefault(…)``), a non-literal store (a computed subscript key, a + dict built from a call), and a SECOND NAME bound to the same dict + (``alias = fields``) each return None rather than the keys seen so far. A + splat with no store in the function at all — the forwarder shape, where ``name`` + is a parameter — is unresolvable too, not vacuously empty. + + The alias direction was the fourth leak in a docstring that claimed three: + ``fields = {"a": 1}`` / ``alias = fields`` / ``alias["customer_email"] = 2`` + resolved to ``{"a"}``, because every store the resolver looks for is spelled on + the OTHER name. Matched narrowly — the assigned value must BE ``Name(name)``, + not merely mention it — so a read (``n = len(fields)``) still resolves.""" + if fn is None: + return None + keys: set[str] = set() + stored = False + for node in ast.walk(fn): + if isinstance(node, (ast.Assign, ast.AnnAssign)): + if isinstance(node.value, ast.Name) and node.value.id == name: + return None + targets = node.targets if isinstance(node, ast.Assign) else [node.target] + for target in targets: + if isinstance(target, ast.Name) and target.id == name: + stored = True + resolved = None if node.value is None else _dict_literal_keys(node.value) + if resolved is None: + return None + keys |= resolved + elif ( + isinstance(target, ast.Subscript) + and isinstance(target.value, ast.Name) + and target.value.id == name + ): + stored = True + if not ( + isinstance(target.slice, ast.Constant) + and isinstance(target.slice.value, str) + ): + return None + keys.add(target.slice.value) + elif isinstance(node, ast.AugAssign): + if isinstance(node.target, ast.Name) and node.target.id == name: + return None + elif ( + isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and isinstance(node.func.value, ast.Name) + and node.func.value.id == name + ): + return None + return keys if stored else None + + +def _enclosing_function_names(tree: ast.AST) -> dict[int, str | None]: + """``id(node) -> the name of the INNERMOST function definition containing it`` + (None at module level). + + ``ast`` nodes carry no parent link and ``ast.walk`` hands them out flat, so the + journal detector — whose splat resolution and whose ``JOURNAL_SPLAT_ALLOW`` key + are both scoped to the function a call sits in — has to build the mapping + itself. Innermost rather than outermost, because that is the scope a ``**name`` + is stored in. + + Deliberately different from the sanctioned-position sets built inside + ``_scan_source``: those use ``ast.walk(fn)``, which descends into nested defs so + a closure inside a sanctioned helper stays sanctioned. Here the innermost answer + is the correct one, and the two uses are not interchangeable.""" + names: dict[int, str | None] = {id(tree): None} + + def descend(node: ast.AST, fn: str | None) -> None: + for child in ast.iter_child_nodes(node): + names[id(child)] = fn + inner = child.name if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef)) else fn + descend(child, inner) + + descend(tree, None) + return names + + +def _enclosing_function_nodes(tree: ast.AST) -> dict[int, ast.AST | None]: + """The node-valued twin of :func:`_enclosing_function_names`, for the splat + resolver, which must WALK the enclosing function rather than name it.""" + nodes: dict[int, ast.AST | None] = {id(tree): None} + + def descend(node: ast.AST, fn: ast.AST | None) -> None: + for child in ast.iter_child_nodes(node): + nodes[id(child)] = fn + inner = child if isinstance(child, (ast.FunctionDef, ast.AsyncFunctionDef)) else fn + descend(child, inner) + + descend(tree, None) + return nodes + + +def _names_rearm_escalation(func: ast.expr, aliases: frozenset[str] = frozenset()) -> bool: + """True when ``func`` spells the re-arm transaction's entry point. + + Qualified and bare spellings are direct matches; ``aliases`` adds ordinary + rename-on-import and assignment bindings. Matching an attribute without checking + its value means an unrelated ``x.rearm_escalation(...)`` also registers — that + false positive is a review prompt naming a real call to a function of that name, + which is the trade every sibling detector in this file makes. + """ + return _names_guarded_verify_call(func, "rearm_escalation", aliases) + + +def _block_exits(body: list[ast.stmt]) -> bool: + """Whether this simple guard body cannot fall through to the re-arm below it.""" + return bool(body) and isinstance(body[-1], (ast.Return, ast.Raise)) + + +def _liveness_call(node: ast.AST) -> bool: + return isinstance(node, ast.Call) and LIVENESS_GATE_MARK in (_called_name(node.func) or "") + + +def _top_level_liveness_bindings(fn: ast.AST, lineno: int) -> set[str]: + """Names bound by an earlier top-level liveness probe in ``fn``.""" + bindings: set[str] = set() + assert isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)) + for stmt in fn.body: + if stmt.lineno >= lineno or not isinstance(stmt, (ast.Assign, ast.AnnAssign)): + continue + value = stmt.value + if value is None or not _liveness_call(value): + continue + targets = stmt.targets if isinstance(stmt, ast.Assign) else [stmt.target] + bindings.update(target.id for target in targets if isinstance(target, ast.Name)) + return bindings + + +def _test_uses_liveness(test: ast.expr, bindings: set[str]) -> bool: + return any( + _liveness_call(node) or (isinstance(node, ast.Name) and node.id in bindings) + for node in ast.walk(test) + ) + + +def _consults_liveness_before(fn: ast.AST | None, lineno: int) -> bool: + """True when a preceding liveness decision blocks fall-through to the re-arm. + + The two real callers keep the gate in their top-level statement sequence: the TUI + calls its boolean helper directly in an ``if`` and the CLI binds ``engine_liveness`` + before testing that result. Requiring a terminating guard body deliberately rejects + an ignored probe, a probe hidden in an uncalled nested function, and one conditional + on an unrelated outer branch. A more deeply factored gate is a review prompt rather + than a silent pass. + """ + if not isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)): + return False + bindings = _top_level_liveness_bindings(fn, lineno) + for stmt in fn.body: + if stmt.lineno >= lineno or not isinstance(stmt, ast.If): + continue + if _block_exits(stmt.body) and _test_uses_liveness(stmt.test, bindings): + return True + return False + + def _scan(): """Single pass over the tree → list of (kind, rel, lineno, line_text).""" findings = [] @@ -574,8 +1335,9 @@ def _scan_source(src: str, rel: str): tree = ast.parse(src, filename=rel) docs = _docstring_node_ids(tree) env_aliases = _env_name_aliases(tree) - verify_command_aliases = _verify_call_aliases(tree, "verify_commands_outcome") - verify_classifier_aliases = _verify_call_aliases(tree, "verify_command_results_outcome") + verify_command_aliases = _call_aliases(tree, "verify_commands_outcome") + verify_classifier_aliases = _call_aliases(tree, "verify_command_results_outcome") + rearm_aliases = _call_aliases(tree, "rearm_escalation") # First positional args of `_run_git(...)` calls — the one position where a # git argv literal feeds the chokepoint instead of bypassing it. Collected up @@ -627,6 +1389,53 @@ def _scan_source(src: str, rel: str): and _names_verify_classifier(call.func, verify_classifier_aliases) } + # String Constants that ARE the task-artifact list rather than a copy of it: the + # elements of `journal.TASK_CYCLE_ARTIFACTS`' own assignment. Skipped by id, so + # the definition needs no allowlist entry and a bare literal elsewhere in the + # same file is still refused (see TASK_ARTIFACT_DEFINITION). + artifact_definition_rel, artifact_definition_name = TASK_ARTIFACT_DEFINITION + artifact_definition_nodes = { + id(const) + for stmt in ast.walk(tree) + if rel == artifact_definition_rel + and isinstance(stmt, (ast.Assign, ast.AnnAssign)) + and stmt.value is not None + and any( + isinstance(target, ast.Name) and target.id == artifact_definition_name + for target in (stmt.targets if isinstance(stmt, ast.Assign) else [stmt.target]) + ) + for const in ast.walk(stmt.value) + if isinstance(const, ast.Constant) + } + + # Everything inside this file's ONE sanctioned task-id composition point, if it + # has one. Same `_function_body_nodes(fn)` shape as the verify sets above — a + # nested def inside the chokepoint is still inside it, a decorator or default is + # not — and empty in every other file, since `.get(rel)` is None there and no + # function is named None. + sanctioned_task_id_nodes = { + id(inner) + for fn in ast.walk(tree) + if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)) + and fn.name == SESSION_TASK_ID_CHOKEPOINT.get(rel) + for inner in _function_body_nodes(fn) + } + + # `return` statements inside a function whose NAME contains `task_id` — the + # second position a mint can hide in, and the one a helper like + # `_sweep_task_id` would use. Matched on the name substring rather than on a + # fixed list: naming the function after what it returns is the whole tell. + task_id_returns = { + id(ret) + for fn in ast.walk(tree) + if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)) and "task_id" in fn.name + for ret in ast.walk(fn) + if isinstance(ret, ast.Return) and ret.value is not None + } + + enclosing_names = _enclosing_function_names(tree) + enclosing_nodes = _enclosing_function_nodes(tree) + def line_at(lineno: int) -> str: return lines[lineno - 1] if 1 <= lineno <= len(lines) else "" @@ -731,6 +1540,94 @@ def line_at(lineno: int) -> str: ): findings.append(("path", rel, node.lineno, line_at(node.lineno))) + # A task-directory artifact name spelled as a literal, outside the one + # assignment that defines the list. Matched by string EQUALITY, never by + # containment: the dev/sweep prompts name `result.json` inside a sentence + # ("…write tasks//result.json, then end your turn"), and flagging prose + # would get the allowlist widened until it meant nothing. Docstrings are + # skipped for the same reason the POSIX-path scan skips them. The finding + # carries `(name, enclosing function)`: the exemption is per-name AND per + # position, so a second literal in another function of an allowlisted file + # is still refused. + if ( + isinstance(node, ast.Constant) + and isinstance(node.value, str) + and id(node) not in docs + and id(node) not in artifact_definition_nodes + and node.value in TASK_CYCLE_ARTIFACTS + ): + findings.append( + ( + "taskartifact", + rel, + node.lineno, + line_at(node.lineno), + (node.value, enclosing_names.get(id(node))), + ) + ) + + # A journal write's field names. Explicit keywords are read straight off the + # call; a `**name` splat is resolved through the literal stores that built it + # in the same function, and emits ONE finding with a None name when it + # cannot be — an unresolvable splat is a hole in the inventory, so it fails + # loud rather than being skipped. Each finding carries + # `(field_or_None, enclosing_function, kind_or_None)`: the benign inventory + # is keyed by field, the splat exemption by position, and the KIND is what + # makes `diagnostics`' kind-scoped routing checkable at all. + # + # The kind is the first positional argument when it is a string literal, and + # None otherwise. None is not "no kind": it is "this scan cannot tell", and + # it emits its own `journalkind` finding so the site fails loud rather than + # being graded against a kind that had to be guessed. + if _is_journal_write(node, rel): + fn_name = enclosing_names.get(id(node)) + first = node.args[0] if node.args else None + kind = ( + first.value + if isinstance(first, ast.Constant) and isinstance(first.value, str) + else None + ) + if kind is None: + findings.append(("journalkind", rel, node.lineno, line_at(node.lineno), fn_name)) + for kw in node.keywords: + if kw.arg is not None: + findings.append( + ( + "journalfield", + rel, + node.lineno, + line_at(node.lineno), + (kw.arg, fn_name, kind), + ) + ) + continue + resolved = ( + _journal_splat_keys(enclosing_nodes.get(id(node)), kw.value.id) + if isinstance(kw.value, ast.Name) + else None + ) + if resolved is None: + findings.append( + ( + "journalfield", + rel, + node.lineno, + line_at(node.lineno), + (None, fn_name, kind), + ) + ) + else: + for field in sorted(resolved): + findings.append( + ( + "journalfield", + rel, + node.lineno, + line_at(node.lineno), + (field, fn_name, kind), + ) + ) + # signal.SIGKILL attribute access (the guarded form is a "SIGKILL" # *string* passed to getattr — not an attribute access — so it's clean) if ( @@ -858,6 +1755,92 @@ def line_at(lineno: int) -> str: ): findings.append(("specanchor", rel, node.lineno, line_at(node.lineno))) + # A session task id COMPOSED rather than obtained from `engine._session_task_id`. + # Two value positions, because those are the two a fifth mint can occupy: a + # binding (`task_id = …`, `SessionSpec(task_id=…)`) and a return from a function + # named for what it returns. A forward — `task_id=spec.task_id`, + # `task_id=str(d["task_id"])`, `task_id=task_id` — reaches neither predicate, + # which is the distinction the whole detector rests on. + # + # Collected into a dict keyed by node id so a value matching through two + # candidate paths (a `.format()` call is both the candidate itself and the + # parent of its arguments) reports once. + # + # NOT COVERED, deliberately, and stated rather than implied. This is a review + # tripwire on the shapes the real mint sites use, not a sandbox; widening it is a + # decision, not a bug fix. Each of these was run through `_scan_source` and + # confirmed silent: + # + # * a store into a dict or an attribute — `record["task_id"] = f"…"`, + # `self.task_id = f"…"`. Neither is a Name binding, a `task_id=` keyword, nor a + # return from a `*task_id*` function. + # * an INTERMEDIATE VARIABLE: `tid = f"{key}-dev-1"` on one line and + # `task_id=tid` on the next. The binding position holds a Name, which is a + # forward as far as this detector can see; following it would mean the + # flow-sensitive resolution `_journal_splat_keys` does for one dict, across + # every string in the file. + # * `"-".join([key, "dev", "1"])` and `fmt % (key, n)` where `fmt` is a Name + # bound to the format string — two more real ways to build a string that + # `_is_str_composition` does not recognise (its own docstring lists them). + minted: dict[int, ast.expr] = {} + + def record_mint(value: ast.expr, *, bare_at_depth: bool) -> None: + for candidate, depth in _mint_candidates(value): + if _is_str_composition(candidate) or ( + _is_bare_str(candidate) and (depth == 0 or bare_at_depth) + ): + minted.setdefault(id(candidate), candidate) + + for node in ast.walk(tree): + if isinstance(node, ast.Assign): + if any(isinstance(t, ast.Name) and t.id == "task_id" for t in node.targets): + record_mint(node.value, bare_at_depth=False) + elif isinstance(node, ast.AnnAssign): + if ( + isinstance(node.target, ast.Name) + and node.target.id == "task_id" + and node.value is not None + ): + record_mint(node.value, bare_at_depth=False) + elif isinstance(node, ast.keyword) and node.arg == "task_id": + record_mint(node.value, bare_at_depth=False) + elif isinstance(node, ast.Return) and id(node) in task_id_returns: + # A function NAMED for the id it returns is already the whole tell, so a + # bare literal stays a finding at depth there (`return safe_segment("x")`) + # — unlike a binding, where a literal argument is the sanctioned + # chokepoint call's own `"dev"` part. + assert node.value is not None # task_id_returns only holds valued returns + record_mint(node.value, bare_at_depth=True) + + for mint in minted.values(): + findings.append( + ( + "taskid", + rel, + mint.lineno, + line_at(mint.lineno), + id(mint) in sanctioned_task_id_nodes, + ) + ) + + # Every `rearm_escalation` CALL, carrying `(enclosing function, gated)` — the two + # facts `REARM_ESCALATION_CALLERS` is an enumeration of. The `def` in `runs.py` is + # not a Call and needs no exemption. + for node in ast.walk(tree): + if isinstance(node, ast.Call) and _names_rearm_escalation(node.func, rearm_aliases): + findings.append( + ( + "rearmcall", + rel, + node.lineno, + line_at(node.lineno), + ( + enclosing_names.get(id(node)), + _consults_liveness_before(enclosing_nodes.get(id(node)), node.lineno), + ), + ) + ) + return findings @@ -1037,6 +2020,371 @@ def test_spec_anchor_detector_stays_silent_on_the_anchored_form(): assert not [f for f in _scan_source(src, "tui/app.py") if f[0] == "specanchor"] +def _task_artifact_offenders(findings) -> list[tuple[str, int, str, str]]: + """The artifact-name literals no declared POSITION covers — the assertion's + whole policy, factored out so it can be graded on synthetic findings rather than + only on today's tree (the file's ``_env_read_offenders`` idiom). + + Both halves of the key bite: the file, then the enclosing function inside it. + Dropping the function half exempts every ``"result.json"`` in + ``adapters/generic.py``, which is what the allowlist's comment already said was + not the case.""" + return [ + (rel, ln, txt, name) + for _, rel, ln, txt, (name, fn) in findings + if name not in TASK_ARTIFACT_LITERAL_ALLOW.get(rel, {}).get(fn, frozenset()) + ] + + +def test_task_cycle_artifacts_named_only_through_the_constant(): + """The task-directory artifact names live in ``journal.TASK_CYCLE_ARTIFACTS``, + not as a literal in each site that touches them. + + Three sites share the list: both adapters clear it in ``start_session`` (a + caller-supplied task_id may be reused, so a silent session must not inherit its + predecessor's outputs) and ``resolve._gather_escalations`` reads it back. They + were three independent literals, and the only parity claim was a sentence in a + test docstring — so a third artifact added to the reader would silently miss + both adapters, which is exactly how ``escalation.json`` reached the reader + before either adapter cleared it. + + The exemption is per-POSITION and per-NAME, never per-file: + ``adapters/generic.py::_result_path`` answers a genuinely single-artifact + question and keeps ``"result.json"``, while ``"escalation.json"`` stays refused + inside it and BOTH names stay refused in every other function of that file. + + ⚠️ What this assertion is worth on today's tree, said as candidly as its DW-66 + sibling says it: almost nothing. There is exactly ONE `taskartifact` finding in + the whole tree and it is allowlisted, so the offender list is empty and would + stay empty with the detector deleted. ``TASK_ARTIFACT_PROBES`` and + ``TASK_ARTIFACT_SCOPE_CASES`` are what grade the detector and the scoping; this + row grades the tree, and the tree is currently clean. + + ⚠️ And what it protects is narrower than "the constant is the list". It refuses + the constant being UN-DONE — a name pulled back out into a literal at any of the + three sites. It does NOT catch the constant being OUT-GROWN: a genuinely new + artifact spelled only in the reader produces no finding at all, because the + detector matches the names the constant already holds. Verified — a + ``(task_dir / "verdict.json")`` added to ``resolve.py`` is silent here, and the + parity it would break is the parity this guard exists for. + + Ablation: respell either adapter's loop as + ``(task_dir / "escalation.json").unlink(missing_ok=True)`` and this reddens + naming that file and line.""" + offenders = _task_artifact_offenders(_of("taskartifact")) + assert offenders == [], ( + "a tasks// artifact named as a bare literal — iterate " + "journal.TASK_CYCLE_ARTIFACTS so the readers and both adapters cannot " + "drift apart on the list:\n" + + "\n".join(f" {rel}:{ln}: {name!r} — {txt.strip()}" for rel, ln, txt, name in offenders) + ) + + +def _session_task_id_offenders(findings) -> list[tuple[str, int, str]]: + """The chokepoint invariant as a filter: a composed task id is sanctioned only + in a ``SESSION_TASK_ID_CHOKEPOINT`` file AND only inside that file's one listed + enclosing function — the file alone is not enough, for the reason the git and + verify-classifier exemptions are not file-wide.""" + return [(rel, ln, txt) for _, rel, ln, txt, at_chokepoint in findings if not at_chokepoint] + + +def test_session_task_id_composed_only_at_the_chokepoint(): + """Every session task id is composed in ``engine._session_task_id`` and nowhere + else. + + The four mint sites (``engine.py`` ×3, ``resolve.py``) all call it and bind or + pass the result; none spells the format. That is what makes + ``_resumable_session``'s resume match byte-identical to what ``_run_session`` + stored, and what carries the ``-g`` re-arm generation discriminator a + hand-rolled fifth mint would omit — silently re-opening #705, correctly + everywhere it was exercised and wrong only on a re-armed run. + + Nothing forbade a fifth. This does: a composition or a bare literal in a + ``task_id`` binding, or returned from a function named for the id it makes, is + refused wherever it is spelled. A FORWARD is not a mint and stays silent — see + ``SESSION_TASK_ID_PROBES`` / ``SESSION_TASK_ID_NON_PROBES`` for that boundary as + rows rather than prose. + + ⚠️ What this assertion grades, precisely — the halves are NOT the same, and + both ablations were run rather than reasoned about: + + * the SANCTION, yes. The chokepoint's own ``return safe_segment(f"…")`` is a + real finding on today's tree, cleared only by its position, so emptying + ``SESSION_TASK_ID_CHOKEPOINT`` reddens this naming ``engine.py:393``. That is + more than the sibling guards' repo-wide assertions can say for themselves. + * the DETECTOR, no. Delete the ``taskid`` emit and this goes green with an empty + finding list — indistinguishable from an invariant that holds. + ``SESSION_TASK_ID_PROBES`` is where that is caught, and the two are not + interchangeable. + + Ablation: respell ``resolve.py``'s mint as + ``task_id=f"{story_key}-resolve-1"`` and this reddens naming that line.""" + offenders = _session_task_id_offenders(_of("taskid")) + assert offenders == [], ( + "a session task id composed outside engine._session_task_id — call that " + "function instead, so the id keeps its whole-composition sanitize and its " + "-g re-arm generation suffix (#705):\n" + + "\n".join(f" {rel}:{ln}: {txt.strip()}" for rel, ln, txt in offenders) + ) + + +def test_rearm_escalation_called_only_behind_a_liveness_gate(): + """``runs.rearm_escalation`` is reached from exactly two places, and each consults + liveness before it. + + ``runs._rearm_commit_landed`` decides whether the re-arm transaction COMMITTED — + and therefore whether to roll the spec back — from ``(generation, phase)`` over the + reloaded task, nothing more. Those two conjuncts are a sufficient identity only + while ``rearm_escalation`` is the sole writer of that run's ``state.json``, and that + model is argued from this enumeration: two callers, each behind a liveness + consultation, with no engine running. A third caller, or either gate deleted, makes + the premise false — and the defect it reopens is DW-79/DW-83's own: a spec left + flipped against a task the run still calls ESCALATED. + + Note what the gate does and does not establish. It proves the engine is not + PROVABLY alive, not that it is dead: ``"alive"`` is refused outright, while + ``"unknown"`` proceeds under ``--force`` in ``cmd_resolve`` and counts as blocking + in the TUI only for a pid-backed run. So this grades the falsifiable half — that + an earlier liveness decision BLOCKS fall-through before the call. The rest of the + model (one control command at a time) is out of scope here and tracked as DW-93. + + ``cli.cmd_resume`` is deliberately absent: it writes this run's ``state.json`` + through ``_resume_paused_run``, so the sole-writer claim must account for it, but it + never re-arms and so is not a call site. Listing it here would make the enumeration + unfalsifiable in the direction that matters. + + ⚠️ What this assertion grades, precisely — the two halves differ, and the + difference is the reason the probe rows below exist: + + * the ENUMERATION, yes, in both directions and with multiplicity. The count is + non-empty on today's tree, so deleting the ``rearmcall`` emit reddens it — unlike the + sibling repo-wide "nothing is flagged" guards, which go green when their detector + dies. Adding a third call, even inside an existing caller, reddens it too. + * the GATE, no. Both sites are gated today, so ``ungated == []`` would survive a + ``_consults_liveness_before`` that always answered ``True`` — including one that + had lost its line-position check, which is the half a late gate would exploit. + ``REARM_CALL_PROBES`` is where that is caught, and the two are not + interchangeable. + + Ablations to run against this row: drop the ``if + self._resolve_blocked_by_liveness(...)`` block from ``tui.TuiApp._do_rearm`` and the + gate half must redden naming ``tui/app.py``; add a call in a third function and the + count comparison must redden.""" + findings = _of("rearmcall") + sites = _rearm_callsite_counts(findings) + declared = Counter(REARM_ESCALATION_CALLERS) + assert sites == declared, ( + "the count of runs.rearm_escalation call sites moved. That enumeration is what " + "runs._rearm_commit_landed's (generation, phase) commit probe argues its " + "sole-writer premise from — a new caller needs that docstring revisited (and " + "DW-93 consulted), not this constant widened:\n" + f" scanned: {sorted(sites.elements())}\n" + f" declared: {sorted(declared.elements())}" + ) + ungated = [(rel, ln, txt) for _, rel, ln, txt, (_, gated) in findings if not gated] + assert ungated == [], ( + "runs.rearm_escalation called without a preceding liveness refusal — the re-arm " + "mutates persisted state for a run it must know is not being driven:\n" + + "\n".join(f" {rel}:{ln}: {txt.strip()}" for rel, ln, txt in ungated) + ) + + +def _journal_field_offenders(findings) -> list[tuple[str, int, str, str]]: + """The routing invariant as a filter, in the two directions a finding can fail: + a field name that neither ``diagnostics`` nor the benign inventory accounts for, + and a ``**splat`` whose keys could not be resolved at a position that has not + declared itself a hole. + + Routing is checked BY NAME first and then BY KIND, mirroring ``_scrub_entry``'s + own order rather than a flattened union of the two. A kind-scoped name — today + only ``target`` — is routed on its own kinds, declared benign on the + ``board-advance-*`` family that carries a sprint status under the same name, and + an offender everywhere else, INCLUDING at a call whose kind the scan could not + resolve. That is the case a by-name union got wrong in the dangerous direction: + ``journal.append("unit-merge-failed", target=branch)`` read as routed.""" + offenders: list[tuple[str, int, str, str]] = [] + for _, rel, ln, txt, (field, fn, kind) in findings: + where = f"{fn}()" if fn else "" + if field is None: + if (rel, fn) not in JOURNAL_SPLAT_ALLOW: + offenders.append((rel, ln, txt, f"unresolvable **splat in {where}")) + continue + if field in JOURNAL_ROUTED_FIELDS or field in JOURNAL_BENIGN_FIELDS: + continue + if kind is not None and ( + field in JOURNAL_KIND_ROUTED_FIELDS.get(kind, frozenset()) + or field in JOURNAL_KIND_BENIGN_FIELDS.get(kind, frozenset()) + ): + continue + on = f"on {kind!r}" if kind is not None else "on a non-literal kind" + offenders.append((rel, ln, txt, f"{field!r} {on} in {where}")) + return offenders + + +def _journal_kind_offenders(findings) -> list[tuple[str, int, str]]: + """Journal writes whose KIND is not a string literal, at a position that has not + declared itself one. Their fields cannot be graded against kind-scoped routing at + all, so — like an unresolvable splat — they fail loud rather than pass by + default.""" + return [ + (rel, ln, txt) + for _, rel, ln, txt, fn in findings + if (rel, fn) not in JOURNAL_DYNAMIC_KIND_ALLOW + ] + + +def test_journal_fields_are_routed_or_declared_benign(): + """Every field name a journal producer SPELLS AT A CALL is either routed by + ``diagnostics`` — by name, or by name-and-kind — or listed in the benign + inventory. + + Two bounds on "every field name a journal producer writes", which is what this + docstring used to claim, and neither is a detail. ``Journal.append`` mints + ``log_task`` and ``log_pos`` itself with ``setdefault``, so no call spells them + and this scan cannot see them (``JOURNAL_SELF_MINTED_FIELDS``; + ``test_journal_append_writes_only_accounted_fields`` is the row that actually + covers them). And a field arriving through a declared ``JOURNAL_SPLAT_ALLOW`` + hole is inventoried there by hand, not observed here. + + Routing is NOT flat, which the earlier wording implied by folding + ``_JOURNAL_KIND_ALIAS_FIELDS`` into one by-name union. ``target`` is aliased on + three merge kinds and deliberately left alone on the ``board-advance-*`` family, + where it carries a sprint status — so it is checked per kind, and + ``journal.append("unit-merge-failed", target=branch)``, a new kind reusing the + name, is refused here rather than sailing through to ``scrub_json``. + + Nothing coupled the producers to the tables, and the tables route by field NAME. + A measured ablation — renaming ``recovery_flow.py``'s ``patch=`` to + ``patch_path=`` — left every row of ``tests/test_diagnostics.py`` green while + the field dropped out of ``_JOURNAL_DROP_FIELDS`` and started shipping in + ``--dump`` output. That is the failure this refuses, and it is a rename rather + than an exotic shape. + + Direction matters, and the reverse would not work. Several routing rows are + deliberately defensive (``paused_story_key``, ``bundle``, ``detail``, + ``suggestion``, ``blocker``, ``stdout_path``) and have no static kwarg producer, + so a "no dead row" assertion would need a large allowlist of CORRECT entries + while catching nothing this direction misses. A rename shows up here as a NEW + unrouted name — which is precisely the measured ablation. + + What the benign inventory claims is narrow and stated plainly on + ``JOURNAL_BENIGN_FIELDS``: it is the set of unrouted names that existed when the + guard landed, not a per-name safety audit. The guard's real assertion is that + the NEXT name cannot appear without someone deciding which side it belongs on. + + A ``**splat`` is resolved through the literal stores that build it; when it + cannot be, the site fails loud unless ``JOURNAL_SPLAT_ALLOW`` declares it a + known hole with a reason. A silently-skipped splat would be a standing hole in + the inventory — the guard would keep passing while new fields arrived through + it. + + Ablation: rename ``recovery_flow.py``'s ``patch=`` to ``patch_path=`` and this + reddens naming the new field.""" + offenders = _journal_field_offenders(_of("journalfield")) + assert offenders == [], ( + "a journal field is neither routed by diagnostics' redaction tables nor " + "declared benign — decide which it is: add a row to the right table in " + "diagnostics.py if it carries an identifier, a path or free text, or list " + "it in JOURNAL_BENIGN_FIELDS if it does not:\n" + + "\n".join(f" {rel}:{ln}: {what} — {txt.strip()}" for rel, ln, txt, what in offenders) + ) + + +def test_journal_kinds_are_literal_or_the_position_is_declared(): + """A journal write whose KIND is not a string literal cannot be graded against + ``diagnostics``' kind-scoped routing, so it fails loud at an undeclared position + — the same stance the guard takes on an unresolvable ``**splat``, and for the + same reason: a site the scan cannot read must not read as clean. + + Seven such writes exist, at four positions, and all four journal only by-name + routed fields today (``JOURNAL_DYNAMIC_KIND_ALLOW`` records which). Declaring one + waives the kind resolution and nothing else: a kind-scoped name at one of them is + still refused by the sibling assertion, because nothing can prove which kind it + lands on. + + Ablation: empty ``JOURNAL_DYNAMIC_KIND_ALLOW`` and this reddens naming all four + positions.""" + offenders = _journal_kind_offenders(_of("journalkind")) + assert offenders == [], ( + "a journal write whose kind is not a string literal, at a position that has " + "not declared itself one — pass a literal kind, or add the position to " + "JOURNAL_DYNAMIC_KIND_ALLOW with what it journals:\n" + + "\n".join(f" {rel}:{ln}: {txt.strip()}" for rel, ln, txt in offenders) + ) + + +def test_journal_field_guard_actually_saw_the_producers(): + """The sibling assertion is an ABSENCE, so it is green both when every field is + accounted for and when the scan stopped finding journal writes at all. This is + the half that cannot be: empty the inventories and the guard must name a real + producer, which proves the scan reached them. + + Also pins the three shapes the scan must not lose — the routed names really are + produced (so ``JOURNAL_ROUTED_FIELDS`` is coupled to live producers rather than + to a copied list), every declared splat hole still exists (so a stale + ``JOURNAL_SPLAT_ALLOW`` entry cannot sit there sanctioning nothing), and every + declared BENIGN name still has a producer. + + That last one is the direction nothing held before. The benign inventory is a + pre-approval list, so a name whose producer was deleted does not just sit there + inertly: it pre-approves a future, unrelated field that happens to reuse the + spelling, with no one making the decision the inventory exists to force. The two + exemptions are the names no CALL can spell — what ``Journal.append`` mints itself + and what arrives through a declared splat hole.""" + findings = _of("journalfield") + produced = {field for _, _, _, _, (field, _, _) in findings if field is not None} + assert len(produced) > 100, f"the scan found only {len(produced)} journal fields" + assert produced & JOURNAL_ROUTED_FIELDS, "no routed field has a static producer" + holes = {(rel, fn) for _, rel, _, _, (field, fn, _) in findings if field is None} + assert holes == set(JOURNAL_SPLAT_ALLOW), ( + "JOURNAL_SPLAT_ALLOW no longer matches the unresolvable splats in the tree; " + f"undeclared: {sorted(holes - set(JOURNAL_SPLAT_ALLOW))}, " + f"stale: {sorted(set(JOURNAL_SPLAT_ALLOW) - holes)}" + ) + unscannable = JOURNAL_SELF_MINTED_FIELDS.union(*JOURNAL_SPLAT_ALLOW.values()) + stale = JOURNAL_BENIGN_FIELDS - produced - unscannable + assert stale == set(), ( + "JOURNAL_BENIGN_FIELDS names fields no producer writes any more — a benign " + "entry outlives its producer as a standing pre-approval for the next field " + "that reuses the name. Delete them, or record where they now come from in " + f"JOURNAL_SPLAT_ALLOW / JOURNAL_SELF_MINTED_FIELDS: {sorted(stale)}" + ) + + +def test_journal_append_writes_only_accounted_fields(tmp_path): + """The static guard reads CALL SITES, and ``Journal.append`` adds two field names + that no call site spells: ``entry.setdefault("log_task", …)`` and + ``entry.setdefault("log_pos", size)``. Both were invisible to it, and ``log_pos`` + was in neither routing set while the guard stayed green — so the sibling's claim + about "every field a producer writes" was false by two names. + + This closes it from the only side that can: RUN an append, read the JSONL line + back, and hold every key it actually contains to the same two inventories. A + third ``setdefault`` cannot be added to ``Journal.append`` without landing in one + of them. + + Ablation: add ``entry.setdefault("log_seq", 0)`` to ``Journal.append`` and this + row reddens naming ``log_seq`` while every static assertion above stays green — + which is the whole point of the row existing beside them.""" + run_dir = tmp_path / "run" + j = Journal(run_dir) + j.set_active_log("1-1-story-dev-1") + j.append("run-start", run_type="stories") + + lines = (run_dir / JOURNAL_FILE).read_text(encoding="utf-8").splitlines() + entry = json.loads(lines[-1]) + minted = set(entry) - {"ts", "kind"} + assert ( + "log_pos" in minted and "log_task" in minted + ), f"Journal.append stopped stamping the pane-log pointer: {sorted(minted)}" + unaccounted = minted - JOURNAL_ROUTED_FIELDS - JOURNAL_BENIGN_FIELDS + assert unaccounted == set(), ( + "Journal.append writes a field name neither routed by diagnostics nor " + "declared benign — the static guard cannot see a field the append mints " + f"itself, so decide which side it belongs on here: {sorted(unaccounted)}" + ) + + def test_no_hardcoded_posix_paths(): """No bare ``/tmp`` / ``/proc`` / ``/dev/null`` literal outside the allowlisted platform-guarded Unity files; each allowed line carries a `# portability:` ack. @@ -1881,6 +3229,903 @@ def test_env_read_allowlist_is_scoped_by_family_not_by_file(label, rel, key, is_ ) +# The artifact-literal detector's probe matrix. Today's tree has exactly ONE +# `taskartifact` finding (generic.py's `_result_path`, allowlisted), so deleting the +# detector branch leaves every tree-wide assertion green — only these rows redden. +# +# Every source below is BUILT BY ITERATING `TASK_CYCLE_ARTIFACTS` rather than by +# indexing it. Two reasons, and the second is the load-bearing one: a renamed +# artifact cannot leave a probe grading a string nothing produces any more, and a +# constant that SHRINKS cannot raise `IndexError` while this module is being +# imported. That error arrives at COLLECTION and takes every guard in this file down +# with it — the POSIX, git, env-read and spec-anchor ones included — which is a very +# large blast radius for a one-line edit in `journal.py`. Iteration degrades to +# fewer rows instead, and `test_artifact_probe_tables_are_not_empty` states the floor. +_ARTIFACT_TUPLE_SRC = ", ".join(f'"{name}"' for name in TASK_CYCLE_ARTIFACTS) +TASK_ARTIFACT_PROBES = [ + *( + (f"unlink-literal:{name}", f'(task_dir / "{name}").unlink(missing_ok=True)\n') + for name in TASK_CYCLE_ARTIFACTS + ), + *( + (f"read-literal:{name}", f'doc = json.loads((d / "{name}").read_text())\n') + for name in TASK_CYCLE_ARTIFACTS + ), + # The re-introduced pair, in the shape the extraction removed: an inline tuple + # in a for-loop, which is how the reader spelled it. + ("inline-tuple-loop", f"for fname in ({_ARTIFACT_TUPLE_SRC}):\n pass\n"), + # A second module re-declaring the constant is a COPY, not the definition — the + # definition skip is keyed to journal.py (see TASK_ARTIFACT_DEFINITION). + ("constant-redeclared-elsewhere", f"TASK_CYCLE_ARTIFACTS = ({_ARTIFACT_TUPLE_SRC})\n"), +] +TASK_ARTIFACT_NON_PROBES = [ + # The detector matches string EQUALITY, never containment: the dev and sweep + # prompts name the artifact inside a sentence, and flagging prose is how a + # tripwire gets allowlisted into meaninglessness. + *( + ( + f"prompt-prose:{name}", + f'PROMPT = "Write your verdict to tasks//{name}, then stop."\n', + ) + for name in TASK_CYCLE_ARTIFACTS + ), + *( + (f"docstring-prose:{name}", f'def f():\n """Reads {name} beside it."""\n return 1\n') + for name in TASK_CYCLE_ARTIFACTS + ), + # A different artifact in the same directory: the guard's claim is about the + # SHARED list, not about every filename a task dir holds. `heartbeat.json` and + # `messages.json` are real siblings that stay outside it (see the constant). + ("sibling-artifact", 'p = task_dir / "prompt.txt"\n'), + ("adapter-owned-sibling", 'p = task_dir / "heartbeat.json"\n'), + # The sanctioned spelling everywhere: iterate the constant. + ( + "iterating-the-constant", + "for artifact in TASK_CYCLE_ARTIFACTS:\n (task_dir / artifact).unlink(missing_ok=True)\n", + ), +] + + +def test_artifact_probe_tables_are_not_empty(): + """The tables above are derived from `TASK_CYCLE_ARTIFACTS` by iteration, which + is what stops a shrunk constant erroring this module's collection — but the same + derivation would quietly EMPTY a parametrized table, and an empty parametrize + passes for exactly the reason an empty scan does. This is that floor, stated as + a requirement rather than left to an `IndexError` nobody would read as one.""" + assert len(TASK_CYCLE_ARTIFACTS) >= 2, ( + "TASK_CYCLE_ARTIFACTS is down to " + f"{list(TASK_CYCLE_ARTIFACTS)}; the scope cases below need one allowlisted " + "name and one refused name to tell a name-scoped exemption from a file-wide one" + ) + assert TASK_ARTIFACT_PROBES and TASK_ARTIFACT_NON_PROBES and TASK_ARTIFACT_SCOPE_CASES + + +@pytest.mark.parametrize( + ("label", "source"), TASK_ARTIFACT_PROBES, ids=[p[0] for p in TASK_ARTIFACT_PROBES] +) +def test_task_artifact_detector_flags_every_literal_spelling(label, source): + """Each way of re-introducing a literal produces a `taskartifact` finding, driven + through the same `_scan_source` the real scan uses.""" + found = [f for f in _scan_source(source, "sweep.py") if f[0] == "taskartifact"] + assert found, f"the {label!r} spelling produced no `taskartifact` finding:\n{source}" + + +@pytest.mark.parametrize( + ("label", "source"), TASK_ARTIFACT_NON_PROBES, ids=[p[0] for p in TASK_ARTIFACT_NON_PROBES] +) +def test_task_artifact_detector_stays_silent_on_lookalikes(label, source): + """The complement: prose that CONTAINS the name, a docstring, a sibling + filename, and the sanctioned loop over the constant are all silent.""" + found = [f for f in _scan_source(source, "sweep.py") if f[0] == "taskartifact"] + assert not found, f"the {label!r} shape was flagged; it is not a copied list:\n{source}" + + +# The artifact exemption's scoping, as rows: `(rel, source, is_offender)`. On the +# real tree a file-scoped allowlist and this position-and-name-scoped one are +# indistinguishable — generic.py's single literal is the only finding — so only +# synthetic sources can tell them apart, and only they carry the drift that does not +# exist yet. Built by iterating the allowlist and the constant, so a rename cannot +# leave a row grading a name nothing declares. +_ALLOWED_IN_GENERIC = TASK_ARTIFACT_LITERAL_ALLOW["adapters/generic.py"]["_result_path"] +TASK_ARTIFACT_SCOPE_CASES = [ + # The sanctioned single-name read: one artifact, named because the question is + # about that one artifact — and named INSIDE the one function that asks it. + *( + ( + f"generic-result-path:{name}", + "adapters/generic.py", + f'def _result_path(self, task_id):\n return self.tasks_dir / task_id / "{name}"\n', + False, + ) + for name in sorted(_ALLOWED_IN_GENERIC) + ), + # …which buys that file NOTHING about the other name. This is the case a + # file-wide allowlist drops on the path alone — and it is the exact drift the + # extraction removed. + *( + ( + f"generic-other-name:{name}", + "adapters/generic.py", + f'def _result_path(self, task_id):\n (task_dir / "{name}").unlink(missing_ok=True)\n', + True, + ) + for name in TASK_CYCLE_ARTIFACTS + if name not in _ALLOWED_IN_GENERIC + ), + # …and it buys no OTHER FUNCTION of that file the allowlisted name either. A + # file-keyed allowlist waves this through on the path alone, which is what the + # allowlist's comment claimed was already impossible and was not. + *( + ( + f"generic-other-function:{name}", + "adapters/generic.py", + f'def start_session(self, spec):\n (task_dir / "{name}").unlink(missing_ok=True)\n', + True, + ) + for name in sorted(_ALLOWED_IN_GENERIC) + ), + # A module-level literal in the allowlisted file has no enclosing function at + # all, so it cannot inherit a function-keyed exemption. + *( + (f"generic-module-level:{name}", "adapters/generic.py", f'STALE = "{name}"\n', True) + for name in sorted(_ALLOWED_IN_GENERIC) + ), + # The twin adapter has no entry at all, so even the allowlisted NAME is refused + # there: nothing in it answers a single-artifact question. + *( + ( + f"opencode-literal:{name}", + "adapters/opencode_http.py", + f'def _result_path(self, task_id):\n return self.tasks_dir / task_id / "{name}"\n', + True, + ) + for name in sorted(_ALLOWED_IN_GENERIC) + ), + # journal.py's own definition is not a copy — skipped by POSITION, so it needs + # no allowlist entry and cannot cover a literal elsewhere in the file. + ( + "journal-definition", + "journal.py", + f"TASK_CYCLE_ARTIFACTS: tuple[str, ...] = ({_ARTIFACT_TUPLE_SRC})\n", + False, + ), + *( + ( + f"journal-bare-literal-beside-it:{name}", + "journal.py", + f"TASK_CYCLE_ARTIFACTS: tuple[str, ...] = ({_ARTIFACT_TUPLE_SRC})\n" + f'STALE = "{name}"\n', + True, + ) + for name in TASK_CYCLE_ARTIFACTS + ), +] + + +@pytest.mark.parametrize( + ("label", "rel", "source", "is_offender"), + TASK_ARTIFACT_SCOPE_CASES, + ids=[c[0] for c in TASK_ARTIFACT_SCOPE_CASES], +) +def test_task_artifact_allowlist_is_scoped_by_position_and_name(label, rel, source, is_offender): + """Being allowlisted buys a file's ONE declared function the artifact NAMES it + declares, and nothing wider. Without this, `TASK_ARTIFACT_LITERAL_ALLOW` could go + back to a set of paths — or to a file -> names map — and every assertion in this + file would stay green.""" + findings = [f for f in _scan_source(source, rel) if f[0] == "taskartifact"] + offenders = _task_artifact_offenders(findings) + assert bool(offenders) is is_offender, ( + f"an artifact literal in {rel} here should " + f"{'be refused' if is_offender else 'be allowed'}:\n{source}" + ) + + +# The task-id detector's probe matrix. Today's tree has exactly one `taskid` +# finding — the chokepoint's own return — so the tree-wide guard would stay green +# with the composition branches deleted; only these rows redden. +SESSION_TASK_ID_PROBES = [ + ("fstring-assignment", 'task_id = f"{task.story_key}-dev-{task.attempt}"\n'), + ("concat-in-keyword", 'spec = SessionSpec(task_id=story + "-review-1", prompt=p)\n'), + ("percent-format", 'task_id = "%s-dev-%d" % (key, seq)\n'), + ("str-format", 'task_id = "{}-dev-1".format(key)\n'), + ("bare-literal-keyword", 'spec = SessionSpec(task_id="triage-1", prompt=p)\n'), + ("annotated-assignment", 'task_id: str = f"{key}-sweep-1"\n'), + # A helper named for what it returns, in both the bare and the wrapped shape — + # the wrapped one is how a fifth mint copied from the chokepoint would look. + ("returned-from-task_id_fn", 'def _sweep_task_id(key):\n return f"{key}-sweep"\n'), + ( + "returned-through-sanitizer", + 'def _sweep_task_id(key):\n return safe_segment(f"{key}-sweep")\n', + ), + # Both branches of a conditional are the same value position. + ("conditional-branch", 'task_id = base if base else f"{key}-dev-1"\n'), + # The chokepoint's own `return safe_segment(f"…")` copied into a BINDING and into + # a KEYWORD — the most likely fifth mint, because it is the sanctioned line moved + # rather than a new idea, and the one that silently drops the `-g` re-arm + # suffix (#705). Both were silent before the binding and keyword legs descended + # through call arguments. + ("binding-wrapped-in-sanitizer", 'task_id = safe_segment(f"{key}-dev-1")\n'), + ( + "keyword-wrapped-in-sanitizer", + 'spec = SessionSpec(task_id=safe_segment(f"{key}-dev-1"), prompt=p)\n', + ), + # …and one level further in, since a wrapper can nest. + ("binding-wrapped-twice", 'task_id = safe_segment(str(f"{key}-dev-1"))\n'), +] +SESSION_TASK_ID_NON_PROBES = [ + # The sanctioned call, and the three FORWARD shapes. A forward is not a mint, + # and this is the distinction the whole detector rests on. + ("chokepoint-call", 'task_id = _session_task_id(key, "dev", 1, gen)\n'), + ("forward-attribute", "handle = SessionHandle(task_id=spec.task_id, native_id=w)\n"), + ("forward-coerced", 'task_id = str(entry.get("task_id", ""))\n'), + ("forward-name", "handle = SessionHandle(task_id=task_id, native_id=w)\n"), + # The parts handed TO the chokepoint are not the id. The binding leg DOES descend + # into call arguments now, so this row is what makes the depth rule load-bearing: + # a bare literal is a mint only at depth 0, or the `"dev"` in every sanctioned + # mint site becomes a finding. + ("chokepoint-call-with-literal-part", 'task_id = _session_task_id(k, "dev", n, gen)\n'), + ( + "chokepoint-keyword-with-literal-part", + 'spec = SessionSpec(task_id=_session_task_id(k, "dev", n, gen), prompt=p)\n', + ), + # The same rule is what keeps the env read silent — `events.py` and both hook + # scripts spell exactly this, and the variable name is a `task_id` binding. + ("env-read", 'task_id = os.environ.get("BMAD_LOOP_TASK_ID")\n'), + ("env-read-with-default", 'task_id = os.environ.get("BMAD_LOOP_TASK_ID", "probe")\n'), + # The shapes the detector deliberately does not reach, pinned as rows so the + # boundary is executed rather than only described in the `NOT COVERED` comment. + ("intermediate-variable", 'tid = f"{key}-dev-1"\nspec = SessionSpec(task_id=tid)\n'), + ("join-composition", 'task_id = "-".join([key, "dev", "1"])\n'), + ("percent-against-a-name", "task_id = fmt % (key, seq)\n"), + # A composition bound to something else entirely — the detector is scoped to the + # `task_id` positions, not to f-strings at large. + ("composition-elsewhere", 'log_name = f"{task_id}.log"\n'), + # A *task_id* function that FORWARDS: its returned literal-keyed subscript is + # not a string Constant in a value position (`tui.data.active_task_id`). + ( + "task_id_fn-forwards", + 'def active_task_id(entries):\n return str(entries[-1]["task_id"])\n', + ), + # Prose is a docstring Expr, never a binding or a return value. + ( + "prose-in-docstring", + 'def f():\n """Ids look like task_id = f\'{key}-dev-1\'."""\n return 1\n', + ), +] + + +@pytest.mark.parametrize( + ("label", "source"), SESSION_TASK_ID_PROBES, ids=[p[0] for p in SESSION_TASK_ID_PROBES] +) +def test_session_task_id_detector_flags_every_mint_shape(label, source): + """Each spelling of a hand-minted id produces a `taskid` finding. `sweep.py` is + an unsanctioned file, so a finding here is also an offender.""" + found = [f for f in _scan_source(source, "sweep.py") if f[0] == "taskid"] + assert found, f"the {label!r} shape produced no `taskid` finding:\n{source}" + + +@pytest.mark.parametrize( + ("label", "source"), + SESSION_TASK_ID_NON_PROBES, + ids=[p[0] for p in SESSION_TASK_ID_NON_PROBES], +) +def test_session_task_id_detector_stays_silent_on_forwards(label, source): + """The complement: the chokepoint call, the three forward shapes, the literal + PARTS handed to the chokepoint, the environment read every hook script uses, a + composition bound elsewhere, and prose are all silent — a guard that flags + forwards would be allowlisted away within a week. + + The last three rows are the DISCLOSED gaps rather than desired silences: + an intermediate variable, `str.join`, and `%` against a Name-bound format + string. They are here so the boundary is executed and cannot drift into a + coverage claim the detector does not make.""" + found = [f for f in _scan_source(source, "sweep.py") if f[0] == "taskid"] + assert not found, f"the {label!r} shape was flagged; it is not a mint:\n{source}" + + +# The task-id exemption's scoping, as rows: `(rel, source, is_offender)`. +SESSION_TASK_ID_SCOPE_CASES = [ + # The real chokepoint, in the shape it ships. + ( + "sanctioned-chokepoint", + "engine.py", + "def _session_task_id(story_key, part, seq, generation):\n" + ' gen = f"-g{generation}" if generation > 0 else ""\n' + ' return safe_segment(f"{story_key}-{part}-{seq}{gen}")\n', + False, + ), + # Being engine.py is NOT enough: it already binds `task_id` three times, so a + # file-wide exemption would leave the invariant unguarded exactly where a fifth + # mint would be written. + ( + "engine-other-function", + "engine.py", + "def _run_sweep(self, task):\n" + ' task_id = f"{task.story_key}-sweep-{task.attempt}"\n' + " return task_id\n", + True, + ), + # The name does not travel: the same function grown in another module cannot + # sanction itself, which is why the sanction pairs the function with the FILE. + ( + "chokepoint-name-in-another-file", + "sweep.py", + "def _session_task_id(story_key, part, seq, generation):\n" + ' return safe_segment(f"{story_key}-{part}-{seq}")\n', + True, + ), + # The measured ablation: resolve.py's mint respelled as an f-string. + ( + "resolve-respelled", + "resolve.py", + 'spec = SessionSpec(task_id=f"{story_key}-resolve-1", prompt=p)\n', + True, + ), + # …and its real spelling stays silent there. + ( + "resolve-real-spelling", + "resolve.py", + 'spec = SessionSpec(task_id=_session_task_id(story_key, "resolve", 1, generation), prompt=p)\n', + False, + ), + # A nested def inside the chokepoint is still inside it (`ast.walk` descends), + # matching how the verify sanctions treat closures. + ( + "nested-inside-chokepoint", + "engine.py", + "def _session_task_id(story_key, part, seq, generation):\n" + " def compose():\n" + ' return f"{story_key}-{part}-{seq}"\n' + " return safe_segment(compose())\n", + False, + ), + # A decorator and a default argument are evaluated where the chokepoint is + # DEFINED, not inside its body, so a mint parked in one is a fifth mint wearing + # the chokepoint's name. The body's own return stays sanctioned in both rows, so + # the offence is the decorator/default alone. ABLATION: restore + # `for inner in ast.walk(fn)` in `sanctioned_task_id_nodes` and both rows FAIL. + ( + "decorator-bypass", + "engine.py", + '@register(SessionSpec(task_id=f"{story_key}-dev-1", prompt=p))\n' + "def _session_task_id(story_key, part, seq, generation):\n" + " return safe_segment(story_key)\n", + True, + ), + ( + "default-arg-bypass", + "engine.py", + "def _session_task_id(\n" + ' story_key, part, seq, generation, *, spec=SessionSpec(task_id=f"{k}-dev-1", prompt=p)\n' + "):\n" + " return safe_segment(story_key)\n", + True, + ), +] + + +@pytest.mark.parametrize( + ("label", "rel", "source", "is_offender"), + SESSION_TASK_ID_SCOPE_CASES, + ids=[c[0] for c in SESSION_TASK_ID_SCOPE_CASES], +) +def test_session_task_id_exemption_is_scoped_to_the_chokepoint(label, rel, source, is_offender): + """Being engine.py buys the file its `_session_task_id` body and nothing wider. + Without this, the sanction could go back to a bare file set and every assertion + here would stay green — the difference only shows up on a fifth mint, which is + the only kind a tripwire is for.""" + findings = [f for f in _scan_source(source, rel) if f[0] == "taskid"] + offenders = _session_task_id_offenders(findings) + assert bool(offenders) is is_offender, ( + f"a composed task id in {rel} here should " + f"{'be refused' if is_offender else 'be allowed'}:\n{source}" + ) + + +# The re-arm caller detector's probe matrix. Today's tree has exactly two `rearmcall` +# findings and BOTH are gated, so the tree-wide guard's `ungated == []` half would stay +# green with the gate logic deleted, or with its line-position check dropped — only +# these rows redden. Each is driven through the real `_scan_source`. +REARM_CALL_PROBES = [ + # (label, source, expected enclosing function, expected `gated`) + ( + "qualified-call-behind-the-gate", + "def cmd_resolve(args):\n" + " live = runs.engine_liveness(run_dir)\n" + ' if live == "alive":\n' + " return\n" + " runs.rearm_escalation(run_dir, story_key)\n", + "cmd_resolve", + True, + ), + # The TUI's spelling, which reaches `runs.liveness` rather than `engine_liveness`. + # This is the row that makes the substring match load-bearing rather than lax. + ( + "tui-spelling-of-the-gate", + "def _do_rearm(self, run_id, run_dir):\n" + " if self._resolve_blocked_by_liveness(run_id, run_dir):\n" + " return\n" + " runs.rearm_escalation(run_dir, story_key)\n", + "_do_rearm", + True, + ), + # A rename-on-import third caller — the alias resolver's first ordinary shape. + ( + "renamed-call-from-import", + "from .runs import rearm_escalation as rearm\n" + "def cmd_something(args):\n" + " if runs.engine_liveness(run_dir):\n" + " return\n" + " rearm(run_dir, story_key)\n", + "cmd_something", + True, + ), + # Assignment aliases are just as callable as import aliases. + ( + "assigned-call-alias", + "handler = runs.rearm_escalation\n" + "def cmd_something(args):\n" + " if runs.engine_liveness(run_dir):\n" + " return\n" + " handler(run_dir, story_key)\n", + "cmd_something", + True, + ), + # Merely reading liveness is not a gate when the result is ignored. + ( + "ignored-liveness-result", + "def cmd_something(args):\n" + " live = runs.engine_liveness(run_dir)\n" + " runs.rearm_escalation(run_dir, story_key)\n", + "cmd_something", + False, + ), + # Nor is a guard hidden in a closure that the caller never invokes. + ( + "uninvoked-nested-guard", + "def cmd_something(args):\n" + " def guard():\n" + " if runs.engine_liveness(run_dir):\n" + " return\n" + " runs.rearm_escalation(run_dir, story_key)\n", + "cmd_something", + False, + ), + # An ungated third caller: the defect this guard exists for. + ( + "no-gate-at-all", + "def cmd_something(args):\n runs.rearm_escalation(run_dir, story_key)\n", + "cmd_something", + False, + ), + # The gate present but BELOW the call, which is not a gate. Without the line + # comparison in `_consults_liveness_before` this row reads as `True` and the whole + # position rule is unheld. + ( + "gate-below-the-call", + "def cmd_something(args):\n" + " runs.rearm_escalation(run_dir, story_key)\n" + " live = runs.engine_liveness(run_dir)\n", + "cmd_something", + False, + ), +] +REARM_CALL_NON_PROBES = [ + # The definition is not a call and needs no exemption. + ("the-definition", "def rearm_escalation(run_dir, story_key=None):\n return None\n"), + # A different function whose name merely starts the same way. + ("similar-name", "def f():\n runs.rearm_escalation_notice(run_dir)\n"), + # A mere mention as a value, not a call. + ("reference-not-a-call", "def f():\n handler = runs.rearm_escalation\n"), +] + + +@pytest.mark.parametrize( + "label,source,fn,gated", REARM_CALL_PROBES, ids=[p[0] for p in REARM_CALL_PROBES] +) +def test_rearm_call_detector_reports_the_site_and_its_gate(label, source, fn, gated): + """Each call shape is found, attributed to its enclosing function, and graded on + whether an earlier liveness guard blocks fall-through. `cli.py` is passed because + nothing in this detector is file-scoped — the enumeration lives in the tree-wide + assertion, not here.""" + found = [f for f in _scan_source(source, "cli.py") if f[0] == "rearmcall"] + assert len(found) == 1, f"the {label!r} shape produced {len(found)} findings:\n{source}" + assert found[0][4] == (fn, gated), f"the {label!r} shape graded as {found[0][4]}" + + +def _rearm_callsite_counts(findings) -> Counter: + """Call-site multiplicity, not just distinct enclosing functions.""" + return Counter((rel, fn) for _, rel, _, _, (fn, _) in findings) + + +def test_rearm_callsite_count_does_not_hide_a_second_call_in_one_function(): + source = ( + "def cmd_resolve(args):\n" + " if runs.engine_liveness(run_dir):\n" + " return\n" + " runs.rearm_escalation(run_dir, first)\n" + " runs.rearm_escalation(run_dir, second)\n" + ) + found = [f for f in _scan_source(source, "cli.py") if f[0] == "rearmcall"] + assert _rearm_callsite_counts(found) == Counter({("cli.py", "cmd_resolve"): 2}) + + +@pytest.mark.parametrize( + "label,source", REARM_CALL_NON_PROBES, ids=[p[0] for p in REARM_CALL_NON_PROBES] +) +def test_rearm_call_detector_stays_silent_on_non_calls(label, source): + """A definition, a reference and a similarly-named neighbour are not call sites. A + detector that flagged these would push noise into the tree-wide enumeration, which + is an equality assertion and so fails on a false positive as loudly as on a miss.""" + found = [f for f in _scan_source(source, "cli.py") if f[0] == "rearmcall"] + assert not found, f"the {label!r} shape produced a `rearmcall` finding:\n{source}" + + +# The journal detector's probe matrix, as `(label, source, expected)` where +# `expected` is the exact set of field names the scan must extract — `None` standing +# for an unresolvable splat. Asserting the SET rather than "something was found" is +# what makes a partial splat resolution fail here instead of quietly under-reporting. +JOURNAL_FIELD_PROBES = [ + # The three receiver spellings in the tree. + ("self-journal", 'self.journal.append("k", story_key=s, patch=p)\n', {"story_key", "patch"}), + ("bare-journal", 'journal.append("k", branch=b)\n', {"branch"}), + ("private-journal", "self._journal.append(kind, plugin=name)\n", {"plugin"}), + # A splat resolved through the literal stores that build it, in both store + # shapes and across the conditional-dict form `engine._run_inner` uses. + ( + "splat-dict-literal", + "def f(self):\n" + ' fields = {"story_key": k, "checkpoint": "story"}\n' + ' self.journal.append("k", **fields)\n', + {"story_key", "checkpoint"}, + ), + ( + "splat-subscript-store", + "def f(self):\n" + ' fields = {"story_key": k}\n' + ' fields["reason"] = "graceful-stop"\n' + ' self.journal.append("k", **fields)\n', + {"story_key", "reason"}, + ), + ( + "splat-conditional-dict", + "def f(self):\n" + ' extras = {"via": stop.via} if stop.via is not None else {}\n' + ' self.journal.append("k", **extras)\n', + {"via"}, + ), + # Explicit keywords and a splat on the SAME call: both halves are collected, so + # a resolvable splat does not shadow its siblings and vice versa. + ( + "splat-mixed-with-explicit", + 'def f(self):\n d = {"a": 1}\n self.journal.append("k", b=2, **d)\n', + {"a", "b"}, + ), + # The unresolvable shapes, each of which must fail LOUD rather than resolve to + # the keys seen so far — a partially-resolved splat is a silent hole. + ( + "splat-computed-key", + 'def f(self):\n d = {}\n d[f"{kind}_path"] = p\n self.journal.append("k", **d)\n', + {None}, + ), + ( + "splat-update-mutation", + 'def f(self):\n d = {"a": 1}\n d.update(b=2)\n self.journal.append("k", **d)\n', + {None}, + ), + ( + "splat-augmented-store", + 'def f(self):\n d = {"a": 1}\n d += other\n self.journal.append("k", **d)\n', + {None}, + ), + ( + "splat-nested-splat", + 'def f(self):\n d = {"a": 1, **other}\n self.journal.append("k", **d)\n', + {None}, + ), + ( + "splat-from-call", + 'def f(self):\n self.journal.append("k", **self._extras(result))\n', + {None}, + ), + ( + "splat-parameter-forwarder", + "def _log(self, kind, **fields):\n self._journal.append(kind, **fields)\n", + {None}, + ), + ("splat-at-module-level", 'journal.append("k", **fields)\n', {None}), + # The fourth direction the resolver has to fail closed in: a SECOND NAME bound to + # the same dict, mutated through the alias. Every store the resolver looks for is + # spelled on `alias`, so the tracked name resolves to `{"a"}` and the new field + # is invisible — a partially-resolved splat reading as green, which is precisely + # what the other three rows exist to prevent. + ( + "splat-aliased-then-mutated", + "def f(self):\n" + ' fields = {"a": 1}\n' + " alias = fields\n" + ' alias["customer_email"] = 2\n' + ' self.journal.append("k", **fields)\n', + {None}, + ), + # …and a plain READ of the dict is not an alias, so it still resolves. + ( + "splat-read-not-aliased", + 'def f(self):\n fields = {"a": 1}\n n = len(fields)\n' + ' self.journal.append("k", **fields)\n', + {"a"}, + ), +] +# The forwarder leg, which needs its own `rel` because `JOURNAL_FORWARDERS` is keyed +# `(file, name)`: `(label, rel, source, expected)`. Without the declaration the plugin +# bus's four `self._log(...)` sites were a wall — the scan saw only the `.append` +# inside `_log`, which is an unresolvable splat, so `rc` and `blocking` reached the +# journal while sitting in neither routing set with the guard green. +JOURNAL_FORWARDER_PROBES = [ + ( + "declared-forwarder-call", + "plugins/bus.py", + 'self._log("plugin-hook", plugin=lp.name, stage=hook.stage, rc=rc, blocking=True)\n', + {"plugin", "stage", "rc", "blocking"}, + ), + # The declaration is keyed by FILE as well as name: a `_log` in another module + # forwards to something else entirely and must stay invisible. + ("forwarder-name-in-another-file", "stories_engine.py", 'self._log("k", rc=rc)\n', set()), + # …and it does not turn every call in the declared file into a journal write. + ("other-call-in-forwarder-file", "plugins/bus.py", 'self._emit("k", rc=rc)\n', set()), +] + + +@pytest.mark.parametrize( + ("label", "rel", "source", "expected"), + JOURNAL_FORWARDER_PROBES, + ids=[p[0] for p in JOURNAL_FORWARDER_PROBES], +) +def test_journal_forwarder_calls_enter_the_inventory(label, rel, source, expected): + """A declared forwarder's CALL SITES are journal writes, so their explicit + keywords are graded like any other producer's — and the declaration is scoped to + the one file that owns the forwarder.""" + found = {f[4][0] for f in _scan_source(source, rel) if f[0] == "journalfield"} + assert found == expected, f"the {label!r} shape resolved to {sorted(found, key=str)}:\n{source}" + + +JOURNAL_FIELD_NON_PROBES = [ + # `.append` on anything that is not a journal handle — the method name alone is + # the most common in the language, so anchoring on the receiver is load-bearing. + ("list-append", "results.append(SessionResult(status=s, stop_seen=True))\n"), + ("attribute-list-append", "self.entries.append(dict(kind=k, story_key=s))\n"), + # A journal write with no fields at all produces nothing to route. + ("kind-only", 'self.journal.append("run-start")\n'), + # Prose naming the call is a Constant, not a Call. + ("prose-in-docstring", 'def f():\n """Calls journal.append(patch=p)."""\n return 1\n'), +] + + +@pytest.mark.parametrize( + ("label", "source", "expected"), + JOURNAL_FIELD_PROBES, + ids=[p[0] for p in JOURNAL_FIELD_PROBES], +) +def test_journal_field_detector_extracts_the_declared_names(label, source, expected): + """The names (and the unresolvable-splat marker) the scan must extract from each + producer shape. Deleting the splat resolver, or letting it return the keys it + managed to see, reddens exactly the rows that describe that behaviour — which + the tree-wide assertion cannot, since it is an absence.""" + found = {f[4][0] for f in _scan_source(source, "sweep.py") if f[0] == "journalfield"} + assert found == expected, f"the {label!r} shape resolved to {sorted(found, key=str)}:\n{source}" + + +@pytest.mark.parametrize( + ("label", "source"), + JOURNAL_FIELD_NON_PROBES, + ids=[p[0] for p in JOURNAL_FIELD_NON_PROBES], +) +def test_journal_field_detector_stays_silent_on_non_journal_appends(label, source): + """The complement: `.append` on a list, on some other attribute, a kind-only + journal write, and prose are all silent. Without this the detector could pass + every row above by flagging every `.append` in the tree.""" + found = [f for f in _scan_source(source, "sweep.py") if f[0] == "journalfield"] + assert not found, f"the {label!r} shape was flagged as a journal field:\n{source}" + + +# The journal offender filter's scoping, as rows: +# `(rel, fn, field, kind, is_offender)`. On the real tree every field is accounted +# for, so a filter that accepted EVERYTHING would look identical — only synthetic +# findings separate them. +JOURNAL_FIELD_SCOPE_CASES = [ + # The measured DW-82 ablation: a routed field renamed by its producer. `patch` + # is routed (dropped); `patch_path` is nothing, and the dump leaks. + ("routed-name", "recovery_flow.py", "_restore", "patch", "stale-restore", False), + ("renamed-off-the-table", "recovery_flow.py", "_restore", "patch_path", "stale-restore", True), + # A declared-benign name stays silent, and a name in neither set is refused + # wherever it appears — the inventory is global, not per-file. + ("declared-benign", "engine.py", "_run_inner", "attempt", "run-start", False), + ("undeclared-new-field", "engine.py", "_run_inner", "customer_email", "run-start", True), + ("undeclared-in-another-file", "sweep.py", "_triage", "customer_email", "sweep-start", True), + # KIND-SCOPED routing, which a flattened by-name union got wrong in the dangerous + # direction. `target` is aliased to a branch on exactly three merge kinds … + ("kind-alias-on-its-own-kind", "worktree_flow.py", "_merge", "target", "unit-merged", False), + # … and is NOT routed on a new kind that reuses the name. Flattened, this passed + # while `_scrub_entry` handed the branch to `scrub_json` verbatim. + ("kind-alias-on-a-new-kind", "worktree_flow.py", "_merge", "target", "unit-merge-failed", True), + # … nor at a call whose kind the scan could not resolve: nothing there can prove + # which kind it lands on, so the name is not routed by default. + ("kind-alias-on-a-non-literal-kind", "worktree_flow.py", "_merge", "target", None, True), + # The board-advance family carries a sprint STATUS under the same name, declared + # benign per kind rather than by widening the by-name set. + ( + "kind-benign-on-its-own-kind", + "engine.py", + "_advance_board", + "target", + "board-advance-carried", + False, + ), + # …and that declaration does not travel to a kind outside the family either. + ( + "kind-benign-on-another-kind", + "engine.py", + "_advance_board", + "target", + "board-advance-invented", + True, + ), + # An unresolvable splat is refused unless its POSITION is a declared hole … + ("undeclared-splat", "sweep.py", "_triage", None, "sweep-start", True), + ("declared-splat-hole", "plugins/bus.py", "_log", None, None, False), + # … and the declaration does not travel: the same function name in another + # module, or another function in the same module, is still a hole. + ("declared-hole-wrong-file", "stories_engine.py", "_log", None, None, True), + ("declared-hole-wrong-function", "plugins/bus.py", "_dispatch", None, None, True), +] + + +@pytest.mark.parametrize( + ("label", "rel", "fn", "field", "kind", "is_offender"), + JOURNAL_FIELD_SCOPE_CASES, + ids=[c[0] for c in JOURNAL_FIELD_SCOPE_CASES], +) +def test_journal_field_offenders_split_routed_benign_and_holes( + label, rel, fn, field, kind, is_offender +): + """The filter's decision, as rows: routed by name, routed on THIS kind, declared + benign globally or on this kind, or an offender — and, for a splat, whether its + `(file, function)` is a declared hole. + + Pins two scopings the real tree cannot show. `JOURNAL_SPLAT_ALLOW` is keyed by + POSITION rather than by function name (no two of its four holes share a name), + and kind-scoped routing is keyed by KIND rather than flattened by name (every + `target` in the tree today sits on a kind that routes or declares it).""" + offenders = _journal_field_offenders( + [("journalfield", rel, 1, f"journal.append(k, {field}=v)", (field, fn, kind))] + ) + assert bool(offenders) is is_offender, ( + f"{rel}::{fn} journalling {field!r} on kind {kind!r} should " + f"{'be refused' if is_offender else 'be allowed'}" + ) + + +# The dynamic-kind declaration's scoping, as rows: `(rel, fn, is_offender)`. +JOURNAL_KIND_SCOPE_CASES = [ + ("declared-position", "plugins/bus.py", "_log", False), + ("declared-position-recovery", "recovery_flow.py", "prune_preserve_refs", False), + # The declaration does not travel by function name, nor by file. + ("undeclared-function-same-file", "plugins/bus.py", "_dispatch", True), + ("declared-name-another-file", "stories_engine.py", "_log", True), + ("undeclared-position", "sweep.py", "_triage", True), +] + + +@pytest.mark.parametrize( + ("label", "rel", "fn", "is_offender"), + JOURNAL_KIND_SCOPE_CASES, + ids=[c[0] for c in JOURNAL_KIND_SCOPE_CASES], +) +def test_journal_kind_declaration_is_scoped_by_position(label, rel, fn, is_offender): + """A non-literal kind is waived at the exact `(file, function)` that declared + itself, and nowhere else — the `JOURNAL_SPLAT_ALLOW` idiom, for the same reason: + a site the scan cannot read must not read as clean because a same-named function + elsewhere is allowed to be unreadable.""" + offenders = _journal_kind_offenders([("journalkind", rel, 1, "journal.append(kind)", fn)]) + assert bool(offenders) is is_offender, ( + f"a non-literal kind in {rel}::{fn} should " + f"{'be refused' if is_offender else 'be allowed'}" + ) + + +def test_journal_kind_probes_flag_a_non_literal_kind(): + """The detector half: a journal write whose kind is a Name, an f-string or a + call emits a `journalkind` finding, and a literal one does not. Without this the + tree-wide assertion is green with the emit deleted.""" + for source in ( + "def f(self):\n self.journal.append(kind, story_key=s)\n", + 'def f(self):\n self.journal.append(f"{family}-pruned", count=n)\n', + "def f(self):\n self.journal.append(_kind_for(x), count=n)\n", + "def f(self):\n self.journal.append(**everything)\n", + ): + assert [f for f in _scan_source(source, "sweep.py") if f[0] == "journalkind"], source + for source in ( + 'def f(self):\n self.journal.append("run-start", story_key=s)\n', + "def f(self):\n results.append(kind)\n", + ): + assert not [f for f in _scan_source(source, "sweep.py") if f[0] == "journalkind"], source + + +def test_journal_routing_tables_are_read_from_diagnostics(): + """`JOURNAL_ROUTED_FIELDS` and `JOURNAL_KIND_ROUTED_FIELDS` are built from the + live `diagnostics` tables, not copied, so the guard cannot drift from the module + it grades. Asserted rather than left to the comment: a future refactor that + inlined the names would pass every other test here while quietly freezing the + routing set.""" + for table in ( + diagnostics._JOURNAL_ALIAS_FIELDS, + diagnostics._JOURNAL_DROP_FIELDS, + diagnostics._JOURNAL_KEYLIST_FIELDS, + ): + assert set(table) <= JOURNAL_ROUTED_FIELDS + assert JOURNAL_KIND_ROUTED_FIELDS == { + kind: frozenset(row) for kind, row in diagnostics._JOURNAL_KIND_ALIAS_FIELDS.items() + } + # …and the kind-scoped names are deliberately NOT in the by-name union. This is + # the assertion that would have caught the flattening: `target` routed by name + # says the board-advance family is covered when `_scrub_entry` does not cover it. + for row in diagnostics._JOURNAL_KIND_ALIAS_FIELDS.values(): + assert not set(row) & JOURNAL_ROUTED_FIELDS, ( + "a kind-scoped field name leaked into the by-name routed union; " + "`_scrub_entry` consults `_JOURNAL_KIND_ALIAS_FIELDS` per kind, so a " + "by-name claim about it is false on every other kind" + ) + # `_JOURNAL_KIND_SCHEMAS` is the FOURTH table `_scrub_entry` consults, and it was + # coupled to this guard by prose alone: deleting its `preference-escalation` row + # left every assertion here green while the fail-closed arm stopped running and + # `customer="AcmeVault"` went back to shipping verbatim (measured). Read it here + # so that cannot recur. + schemas = diagnostics._JOURNAL_KIND_SCHEMAS + assert schemas, ( + "`_JOURNAL_KIND_SCHEMAS` is empty — `_scrub_entry`'s fail-closed arm is now " + "unreachable and every off-schema key falls back to `scrub_json`" + ) + # The kind whose keys are LLM-authored is the reason the table exists, and + # `JOURNAL_SPLAT_ALLOW`'s comment for `engine.py::_review_and_commit` names this + # table as the mechanism that covers that hole. Pinned rather than trusted: a + # comment naming a mechanism that is not there is the failure this file exists + # to refuse. + assert "preference-escalation" in schemas + assert ( + schemas["preference-escalation"] == JOURNAL_SPLAT_ALLOW[("engine.py", "_review_and_commit")] + ), ( + "the declared schema and the splat inventory that cites it disagree — one " + "of the two was edited alone" + ) + for kind, names in schemas.items(): + # An empty declared set would collapse a record ENTIRELY, presence-marking + # every field including the ones the record is read for. Never the intent: + # a kind with nothing worth showing should not be in this table at all. + assert names, f"{kind} declares an empty schema, which collapses its whole record" + # Every declared name is accounted for on the guard's side too, so a schema + # can neither name a field nothing produces nor quietly introduce one that + # bypassed the routed/benign decision. `type` and `severity` reach the + # journal only through the allowlisted splat, so the inventory there is + # where they are declared. + unaccounted = names - JOURNAL_ROUTED_FIELDS - JOURNAL_BENIGN_FIELDS + unaccounted -= frozenset().union(*JOURNAL_SPLAT_ALLOW.values()) + assert unaccounted == set(), ( + f"{kind}'s declared schema names fields the guard does not account for: " + f"{sorted(unaccounted)}" + ) + # A declared name must not also be kind-aliased on the same kind: the alias + # arm runs FIRST, so such a name would never reach the schema arm and the + # declaration would be a dead letter that reads as live. + assert not names & JOURNAL_KIND_ROUTED_FIELDS.get(kind, frozenset()) + + # and the sets are disjoint: a routed name must never also be declared + # benign, which would make the routing row unfalsifiable from this side. + assert not JOURNAL_ROUTED_FIELDS & JOURNAL_BENIGN_FIELDS + for kind, row in JOURNAL_KIND_ROUTED_FIELDS.items(): + assert not row & JOURNAL_KIND_BENIGN_FIELDS.get(kind, frozenset()) + assert not row & JOURNAL_BENIGN_FIELDS + + def test_guard_actually_scanned_files(): """Sanity: the scan walked a non-trivial number of files (catches a broken SRC root silently passing every assertion).""" diff --git a/tests/test_resolve.py b/tests/test_resolve.py index 1a55a99f..4cbcb129 100644 --- a/tests/test_resolve.py +++ b/tests/test_resolve.py @@ -1,16 +1,17 @@ """Escalation-resolution: context build, re-arm, spec field writer, session.""" import json +import os import sys from pathlib import Path import pytest import yaml -from conftest import escalated_run, git +from conftest import escalated_run, git, json_recursion_payload from bmad_loop import devcontract, platform_util, resolve, runs, verify from bmad_loop.engine import _session_task_id -from bmad_loop.journal import load_state, save_state +from bmad_loop.journal import TASK_CYCLE_ARTIFACTS, load_state, save_state from bmad_loop.model import ( PAUSE_ESCALATION, Phase, @@ -92,6 +93,22 @@ def _escalated_run( return run.run_dir, run.state, run.task +def _context(state, run_dir, story_key, *, isolation): + """`build_context`'s Path alone, for the ~30 rows that assert on `context.json`. + + `build_context` returns `(path, withheld, unreadable)` since DW-11, and both counts + are OPERATOR-facing numbers the CLI prints — no row here is about them. Routing + every Path-only caller through one unpack pins the arity for all of them at once: + grow the tuple a fourth member and this helper fails, rather than every row silently + binding a longer tuple to `path` (which is what a bare `path, _ = ...` at each + site would do). The rows that ARE about the counts call `resolve.build_context` + directly, so neither number is ever produced by this helper.""" + path, _withheld, _unreadable = resolve.build_context( + state, run_dir, story_key, isolation=isolation + ) + return path + + # ------------------------------------------------------------ set_frontmatter_field # # `set_frontmatter_status`'s own tests live in tests/test_frontmatter.py, next to @@ -531,7 +548,7 @@ def test_build_context_gathers_critical_escalations(tmp_path): ), encoding="utf-8", ) - path = resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="") + path = _context(state, run_dir, "6-4-cli-list-command", isolation="") ctx = json.loads(path.read_text(encoding="utf-8")) assert ctx["story_key"] == "6-4-cli-list-command" assert ctx["spec_file"] == spec.as_posix() @@ -589,7 +606,7 @@ def test_build_context_absolutizes_an_isolated_units_worktree_relative_spec(tmp_ run_dir, state, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(wt)) monkeypatch.chdir(tmp_path) # what the resolve session actually runs from - path = resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="worktree") + path = _context(state, run_dir, "6-4-cli-list-command", isolation="worktree") ctx = json.loads(path.read_text(encoding="utf-8")) assert Path(ctx["spec_file"]).is_absolute() # the worktree's copy, not the main checkout's twin — compared as posix, which is @@ -611,24 +628,25 @@ def test_build_context_spec_file_is_none_without_a_task_or_a_spec(tmp_path): run_dir, state, _ = _escalated_run(tmp_path, spec_file=None, worktree_path=str(wt)) ctx = json.loads( - resolve.build_context( - state, run_dir, "6-4-cli-list-command", isolation="worktree" - ).read_text(encoding="utf-8") + _context(state, run_dir, "6-4-cli-list-command", isolation="worktree").read_text( + encoding="utf-8" + ) ) assert ctx["spec_file"] is None # task present, spec-less escalation assert "no-such-story" not in state.tasks ctx = json.loads( - resolve.build_context(state, run_dir, "no-such-story", isolation="worktree").read_text( - encoding="utf-8" - ) + _context(state, run_dir, "no-such-story", isolation="worktree").read_text(encoding="utf-8") ) assert ctx["spec_file"] is None # no task at all + # ... and the escalation gather degrades on the same absence rather than + # dereferencing the missing task (`_gather_escalations` returns [] up front). + assert ctx["escalations"] == [] def test_build_context_no_session_files(tmp_path): run_dir, state, _ = _escalated_run(tmp_path, with_session=False) - path = resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="") + path = _context(state, run_dir, "6-4-cli-list-command", isolation="") ctx = json.loads(path.read_text(encoding="utf-8")) assert ctx["escalations"] == [] assert ctx["paused_reason"].startswith("CRITICAL") @@ -643,25 +661,25 @@ def test_build_context_restore_supported_signal(tmp_path): run_dir, state, task = _escalated_run(tmp_path, spec_file="/abs/spec.md", with_session=False) key = "6-4-cli-list-command" - path = resolve.build_context(state, run_dir, key, isolation="") + path = _context(state, run_dir, key, isolation="") assert json.loads(path.read_text(encoding="utf-8"))["restore_supported"] is True - path = resolve.build_context(state, run_dir, key, isolation="worktree") + path = _context(state, run_dir, key, isolation="worktree") assert json.loads(path.read_text(encoding="utf-8"))["restore_supported"] is False task.worktree_path = str(tmp_path / "wt") # recorded worktree execution - path = resolve.build_context(state, run_dir, key, isolation="") + path = _context(state, run_dir, key, isolation="") assert json.loads(path.read_text(encoding="utf-8"))["restore_supported"] is False task.worktree_path = "" task.spec_file = None # spec-less escalation: a restored patch has no review to resume - path = resolve.build_context(state, run_dir, key, isolation="") + path = _context(state, run_dir, key, isolation="") assert json.loads(path.read_text(encoding="utf-8"))["restore_supported"] is False task.spec_file = "/abs/spec.md" state.source = "stories" task.sentinel_kind = "missing-prd" # pre-planning wedge: nothing attempted to restore - path = resolve.build_context(state, run_dir, key, isolation="") + path = _context(state, run_dir, key, isolation="") assert json.loads(path.read_text(encoding="utf-8"))["restore_supported"] is False @@ -672,7 +690,7 @@ def test_build_context_sanitizes_dirty_story_key(tmp_path): dirty = "6-4:cli?list" seg = safe_segment(dirty) assert seg != dirty - path = resolve.build_context(state, run_dir, dirty, isolation="") + path = _context(state, run_dir, dirty, isolation="") assert path.parent.name == seg ctx = json.loads(path.read_text(encoding="utf-8")) assert ctx["story_key"] == dirty @@ -686,7 +704,7 @@ def test_rearm_flips_phase_and_spec_status(tmp_path): spec = tmp_path / "spec.md" spec.write_text(SPEC, encoding="utf-8") run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) - key = runs.rearm_escalation(run_dir, isolated_redrive=False) + key = runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert key == "6-4-cli-list-command" state = load_state(run_dir) task = state.tasks[key] @@ -710,7 +728,7 @@ def test_rearm_strips_stale_terminal_section(tmp_path): encoding="utf-8", ) run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) text = spec.read_text(encoding="utf-8") assert "Auto Run Result" not in text and "names not unique" not in text assert verify.read_frontmatter(spec)["status"] == "ready-for-dev" @@ -753,7 +771,7 @@ def test_rearm_warns_when_an_isolated_tasks_spec_writes_cannot_reach_the_redrive tmp_path, spec_file=str(spec), worktree_path=str(tmp_path / "wt" / "u1") ) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) (rec,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert rec["story_key"] == "6-4-cli-list-command" @@ -767,6 +785,55 @@ def test_rearm_warns_when_an_isolated_tasks_spec_writes_cannot_reach_the_redrive assert next_step +def test_rearm_completes_on_an_unreachable_spec_it_could_not_capture(tmp_path, monkeypatch): + """The preimage refusal is gated on the SAME pair as the flip's refusal, so a spec the + re-drive does not read keeps warn-and-continue. + + This is the isolated shape the row above builds: `task_spec_path` anchors the writes on + the mount, and a re-armed task's mount is discarded before the re-drive reads anything, + so the readable file is the copy that is destroyed. `rearm-spec-write-unreachable` has + already recorded that fact by the time the preimage is captured. + + Add one transient `EIO` on the first `read_bytes` of that spec and, gated on + `is_file()` ALONE, the re-arm aborted — demanding that the operator repair a file the + re-drive never opens, over a remedy that cannot change what it reads, at the cost of + the interactive resolve session. The unreadable preimage is an OBSERVATION on this + shape, and observations degrade: `spec_before` stays `None` and the re-arm completes. + + Its sibling `tests/test_runs.py::test_rearm_refuses_a_spec_whose_bytes_it_could_not_capture` + holds the other half — on a REACHABLE spec the same fault still refuses, because there + the write it is about to publish is the one the re-drive will read. + + Ablation: drop the `write_reaches_the_redrive` conjunct and this reddens with the + `RearmError` the reachable row expects, while that row stays green. + """ + _resolve_repo(tmp_path) + spec = tmp_path / "spec.md" + spec.write_text(SPEC, encoding="utf-8") + run_dir, _, _ = _escalated_run( + tmp_path, spec_file=str(spec), worktree_path=str(tmp_path / "wt" / "u1") + ) + real_read_bytes = Path.read_bytes + failed_once = [] + + def flaky(self): + if self == spec and not failed_once: + failed_once.append(1) + raise OSError(5, "Input/output error") + return real_read_bytes(self) + + monkeypatch.setattr(Path, "read_bytes", flaky) + + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) + + assert failed_once # the fault really did land on the capture + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.PENDING + # ...and the record that DOES describe this shape is still the one written + (rec,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] + assert rec["spec_file"] == str(spec) + assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-aborted"] == [] + + def test_rearm_does_not_warn_about_unreachable_writes_without_a_worktree(tmp_path): """The control for the row above: the in-place case is where those writes DO land, so a record there would fire on every ordinary re-arm.""" @@ -775,7 +842,7 @@ def test_rearm_does_not_warn_about_unreachable_writes_without_a_worktree(tmp_pat spec.write_text(SPEC, encoding="utf-8") run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] == [] @@ -843,9 +910,9 @@ def test_rearm_journals_a_status_flip_that_silently_did_nothing(tmp_path, shape) if shape == "no-frontmatter": with pytest.raises(runs.RearmError, match="no frontmatter `status:`"): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) else: - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) records = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-flip-skipped"] if shape == "already-at-target": @@ -876,7 +943,7 @@ def test_rearm_journals_a_status_flip_that_silently_did_nothing(tmp_path, shape) def test_rearm_journals_event(tmp_path): run_dir, _, _ = _escalated_run(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) journal = (run_dir / "journal.jsonl").read_text(encoding="utf-8") assert "story-escalation-resolved" in journal @@ -895,7 +962,7 @@ def test_rearm_advances_baseline_to_resolved_head(project): # a file the resolve session (or the user) left untracked must enter the # snapshot, so the redrive reset treats it as pre-existing, not run-created (root / "leftover.txt").write_text("keep me\n", encoding="utf-8") - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.baseline_commit == git(root, "rev-parse", "HEAD") assert task.baseline_commit != old_head @@ -914,7 +981,7 @@ def boom(repo): raise verify.GitError("simulated failure") monkeypatch.setattr(runs.verify, "untracked_files", boom) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.baseline_commit == "abc123" assert task.baseline_untracked is None @@ -924,7 +991,7 @@ def test_rearm_keeps_stale_baseline_outside_a_repo(tmp_path): # best-effort contract: a project dir that is not a git repo (or a broken # one) must not make re-arm fail — the old baseline simply stands run_dir, _, _ = _escalated_run(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.baseline_commit == "abc123" @@ -944,7 +1011,7 @@ def test_rearm_journals_a_failed_baseline_advance(tmp_path): """ run_dir, _, _ = _escalated_run(tmp_path) # tmp_path is not a git repo - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-advance-failed"] assert entry["story_key"] == "6-4-cli-list-command" @@ -972,7 +1039,7 @@ def boom(repo): monkeypatch.setattr(runs.verify, "untracked_files", boom) with pytest.raises(MemoryError): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) @pytest.mark.parametrize("restore", [None, "artifacts/attempt.patch"]) @@ -996,7 +1063,9 @@ def boom(repo): raise verify.GitError("simulated failure") monkeypatch.setattr(runs.verify, "untracked_files", boom) - runs.rearm_escalation(run_dir, restore_patch=restore, isolated_redrive=False) + runs.rearm_escalation( + run_dir, restore_patch=restore, isolated_redrive=False, resolution_recorded=True + ) fm = verify.read_frontmatter(spec) assert fm["baseline_revision"] == old_head # NOT re-stamped with the stale sha @@ -1021,7 +1090,7 @@ def test_rearm_bumps_the_task_generation(tmp_path): before = load_state(run_dir).tasks["6-4-cli-list-command"] assert before.generation == 0 and len(before.sessions) == 1 - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.generation == 1 @@ -1029,7 +1098,7 @@ def test_rearm_bumps_the_task_generation(tmp_path): assert len(task.sessions) == 1 # the audit trail survives the re-arm save_state(run_dir, _rearmable(run_dir)) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert load_state(run_dir).tasks["6-4-cli-list-command"].generation == 2 @@ -1054,7 +1123,7 @@ def test_rearm_advances_the_baseline_in_the_code_tree(tmp_path): git(code, "commit", "-q", "-m", "resolution fixture") (code / "leftover.txt").write_text("keep me\n") - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.baseline_commit == git(code, "rev-parse", "HEAD") != head @@ -1112,7 +1181,9 @@ def test_rearm_reads_stale_restore_residue_from_the_code_tree(tmp_path): git(code, "commit", "-q", "-m", "resolution fixture") new_head = git(code, "rev-parse", "HEAD") - runs.rearm_escalation(run_dir, isolated_redrive=False) # from scratch: the latch is dropped + runs.rearm_escalation( + run_dir, isolated_redrive=False, resolution_recorded=True + ) # from scratch: the latch is dropped task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.baseline_commit == new_head @@ -1153,7 +1224,7 @@ def test_rearm_falls_back_to_project_when_no_code_root_was_recorded(tmp_path): (run_dir / "state.json").write_text(json.dumps(raw), encoding="utf-8") assert load_state(run_dir).repo_root == "" - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert load_state(run_dir).tasks["6-4-cli-list-command"].baseline_commit == head @@ -1200,7 +1271,7 @@ def test_rearm_writes_the_worktree_spec_not_the_main_checkouts_copy(monkeypatch, run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(wt)) monkeypatch.chdir(tmp_path) # what `bmad-loop resolve` actually runs from - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) fm = verify.read_frontmatter(wt / rel) assert fm["status"] == "ready-for-dev" # the flip landed in the WORKTREE @@ -1237,7 +1308,7 @@ def test_rearm_journals_a_skip_when_the_recorded_spec_is_not_readable(tmp_path): run_dir, _, _ = _escalated_run(tmp_path, spec_file="wt/_bmad-output/specs/gone.md") runs.rearm_escalation( - run_dir, isolated_redrive=False + run_dir, isolated_redrive=False, resolution_recorded=True ) # must not raise: the flip's no-op is not a refusal kinds = _kinds(run_dir) @@ -1277,7 +1348,7 @@ def test_rearm_records_an_unreachable_spec_even_when_the_advance_failed(tmp_path _resolve_repo(tmp_path) run_dir, _, _ = _escalated_run(tmp_path, spec_file="wt/_bmad-output/specs/gone.md") - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) kinds = _kinds(run_dir) (skipped,) = [e for e in kinds if e["kind"] == "rearm-baseline-restamp-skipped"] @@ -1305,7 +1376,7 @@ def test_rearm_restamps_normally_when_the_spec_resolves(tmp_path): spec.write_text("---\nstatus: 'escalated'\nbaseline_revision: 'old'\n---\n\nbody\n") run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) kinds = _kinds(run_dir) assert [e for e in kinds if e["kind"] == "rearm-baseline-restamp-skipped"] == [] @@ -1334,7 +1405,7 @@ def test_rearm_clears_sentinel_preserving_a_copy(tmp_path): tmp_path, spec_file=str(sentinel), source="stories", sentinel_kind="unresolved" ) - returned = runs.rearm_escalation(run_dir, isolated_redrive=False) + returned = runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert returned == key # sentinel deleted from disk, a copy preserved under the run dir @@ -1374,7 +1445,7 @@ def test_rearm_non_sentinel_spec_still_flips_status(tmp_path): # detected as a sentinel) → status-flip, not delete. run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec), source="stories") - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert spec.is_file() # not deleted assert verify.read_frontmatter(spec)["status"] == "ready-for-dev" assert load_state(run_dir).tasks[key].spec_file == str(spec) # kept @@ -1394,7 +1465,7 @@ def test_rearm_sentinel_named_spec_never_detected_is_not_deleted(tmp_path): # stories mode, but sentinel_kind unset — the run never classified it as a sentinel run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec), source="stories") - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert spec.is_file() # NOT deleted despite the sentinel-shaped name assert verify.read_frontmatter(spec)["status"] == "ready-for-dev" assert load_state(run_dir).tasks[key].spec_file == str(spec) # kept @@ -1411,7 +1482,7 @@ def test_rearm_sprint_spec_named_like_a_sentinel_is_not_deleted(tmp_path): spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nreal work\n", encoding="utf-8") run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec)) # sprint-status source - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert spec.is_file() # NOT deleted despite the sentinel-shaped name assert verify.read_frontmatter(spec)["status"] == "ready-for-dev" # flipped like any spec assert load_state(run_dir).tasks[key].spec_file == str(spec) # kept @@ -1438,7 +1509,10 @@ def test_rearm_rejects_restore_patch_on_a_sentinel(tmp_path): with pytest.raises(runs.RearmError, match="sentinel"): runs.rearm_escalation( - run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, ) assert sentinel.is_file() # nothing deleted, copy NOT preserved — no clear happened @@ -1460,7 +1534,10 @@ def test_rearm_rejects_restore_patch_without_a_spec_file(tmp_path): with pytest.raises(runs.RearmError, match="no recorded spec file"): runs.rearm_escalation( - run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, ) task = load_state(run_dir).tasks["6-4-cli-list-command"] @@ -1469,7 +1546,7 @@ def test_rearm_rejects_restore_patch_without_a_spec_file(tmp_path): assert not (run_dir / "journal.jsonl").exists() # nothing journaled runs.rearm_escalation( - run_dir, isolated_redrive=False + run_dir, isolated_redrive=False, resolution_recorded=True ) # a from-scratch re-arm remains available assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.PENDING @@ -1487,14 +1564,20 @@ def test_rearm_rejects_restore_patch_for_a_worktree_executed_task(tmp_path): with pytest.raises(runs.RearmError, match="worktree-isolation"): runs.rearm_escalation( - run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=True + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=True, + resolution_recorded=True, ) task = load_state(run_dir).tasks["6-4-cli-list-command"] assert task.phase == Phase.ESCALATED # nothing mutated; still armed for a re-resolve assert task.restore_patch is None # a from-scratch re-arm of the same task is unaffected — the guard is latch-only - assert runs.rearm_escalation(run_dir, isolated_redrive=True) == "6-4-cli-list-command" + assert ( + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) + == "6-4-cli-list-command" + ) def test_validate_restore_latch_passes_a_clean_in_place_escalation(tmp_path): @@ -1521,7 +1604,12 @@ def test_rearm_restore_patch_on_a_real_stories_spec_is_allowed(tmp_path): spec.write_text("---\nstatus: blocked\n---\n\n## Intent\n\nx\n", encoding="utf-8") run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec), source="stories") - runs.rearm_escalation(run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False) + runs.rearm_escalation( + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, + ) task = load_state(run_dir).tasks[key] assert task.phase == Phase.PENDING assert task.restore_patch == "artifacts/attempt.patch" @@ -1561,7 +1649,12 @@ def test_rearm_restore_patch_restamps_spec_baseline(tmp_path): git(tmp_path, "commit", "-q", "-m", "resolution fixture") new_head = git(tmp_path, "rev-parse", "HEAD") - runs.rearm_escalation(run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False) + runs.rearm_escalation( + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, + ) fm = verify.read_frontmatter(spec) assert fm["baseline_revision"] == new_head # step-04 diffs from the ADVANCED baseline @@ -1611,7 +1704,7 @@ def test_rearm_restamps_spec_baseline_on_the_from_scratch_leg_too(tmp_path): old_head = _resolve_repo(tmp_path) run_dir, spec, new_head = _escalated_spec_run(tmp_path, old_head) - runs.rearm_escalation(run_dir, isolated_redrive=False) # no restore + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) # no restore fm = verify.read_frontmatter(spec) assert fm["baseline_revision"] == new_head @@ -1639,10 +1732,15 @@ def test_rearm_restores_the_spec_when_the_baseline_restamp_aborts(tmp_path): to `ready-for-dev` and stripped of the `## Auto Run Result` section the next resolve session reads as its context — the one edit nothing else records. - Ablation: drop the `_restore_rearmed_spec(...)` call from the re-stamp's except arm - and this reddens on the byte comparison (the status flip and the strip both stand), - while the `RearmError` and the ESCALATED phase keep passing — which is exactly why - those two alone do not grade this. + The undo is no longer written into the re-stamp's own `except` arm: the whole window + from the first spec write to `save_state` is one transaction, and its guard rolls the + spec back for every fault that escapes — this one included. What the arm still owns is + the `RearmError` and its remedy. + + Ablation: delete the `except BaseException` arm from `rearm_escalation` and this + reddens on the byte comparison (the status flip and the strip both stand), while the + `RearmError` and the ESCALATED phase keep passing — which is exactly why those two + alone do not grade this. """ old_head = _resolve_repo(tmp_path) spec = tmp_path / "spec.md" @@ -1662,7 +1760,7 @@ def test_rearm_restores_the_spec_when_the_baseline_restamp_aborts(tmp_path): git(tmp_path, "commit", "-q", "-m", "resolution fixture") with pytest.raises(runs.RearmError, match="baseline_revision"): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert spec.read_bytes() == before # flip AND strip both undone # nothing was persisted either, so the escalation is still armed for a corrected spec @@ -1690,9 +1788,11 @@ def test_rearm_restores_the_spec_when_the_result_strip_faults(tmp_path, monkeypa reddens the flip first and leaves nothing to restore. The injection stands in for the faults above, which are real and are exactly what the atomic writers exist for. - Ablation: drop the `_restore_rearmed_spec(...)` call from that arm and this reddens on - the byte comparison alone — the `RearmError` and the ESCALATED phase both still pass, - since the flip landing is precisely what neither observes. Both of those assertions + Ablation: delete the `except BaseException` arm from `rearm_escalation` — the + transaction guard that now performs this undo, in place of the per-arm call this test + used to grade — and it reddens on the byte comparison alone. The `RearmError` and the + ESCALATED phase both still pass, since the flip landing is precisely what neither + observes. Both of those assertions are load-bearing for that claim, so both stay in THIS test: an isolated sibling row was once inserted between them and silently adopted the phase check, leaving this docstring citing an assertion the test no longer made. @@ -1713,7 +1813,7 @@ def boom(spec_path, *, confine_root): monkeypatch.setattr(runs.devcontract, "strip_auto_run_result", boom) with pytest.raises(runs.RearmError, match="No space left on device"): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert spec.read_bytes() == before # the published flip is rolled back assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.ESCALATED @@ -1727,20 +1827,21 @@ def test_rearm_restores_an_isolated_tasks_spec_that_sits_outside_the_worktree( An absolute `spec_file` beside a set `worktree_path` means the spec is lexically OUTSIDE the mount (`model._serialized_worktree_path` keeps a path verbatim exactly when `relative_to(worktree_path)` raises) — the shape a shared artifact directory - produces. `_restore_rearmed_spec` calls `atomic_write_bytes_confined` DIRECTLY, so - a `confine_root` naming the worktree does not merely degrade the write the way the - three `_atomic_write_spec` writers do: it raises `UnconfinedWriteError`, which the - arm re-raises as "cannot restore ...". The operator was then left with the exact - state the undo exists to prevent — a spec carrying this re-arm's status flip and - stripped of its `## Auto Run Result`, on a story the run still calls ESCALATED — - plus a second error masking the first. - - `task_spec_root` now answers the project for that shape, which CAN confine the - spec, so the restore lands and the original fault is the one that surfaces. - - Ablation: revert `task_spec_root` to `Path(task.worktree_path or state.project)` - and this reddens twice — the `match=` fails on "cannot restore ... UnconfinedWrite - Error", and the byte comparison fails behind it. + produces. `task_spec_root` answers the PROJECT there rather than the mount, which CAN + confine this spec, so the undo takes its confined arm and lands, and the original + fault is the one that surfaces — instead of an `UnconfinedWriteError` re-raised as + "cannot restore ..." over a spec left carrying this re-arm's status flip and stripped + of its `## Auto Run Result`, on a story the run still calls ESCALATED. + + Ablation: revert `task_spec_root` to `Path(task.worktree_path or state.project)` AND + make `_restore_rearmed_spec` take `atomic_write_bytes_confined` unconditionally; this + then reddens twice, on the `match=` and on the byte comparison behind it. Both halves + are needed because either one alone now rescues the write, and that redundancy is + deliberate — the root moved for this shape (graded directly by + `test_task_spec_root_yields_the_project_when_the_worktree_cannot_confine_the_spec`) + and the undo later gained the same lexical arm its three sibling writers have, which + is what carries a spec outside BOTH roots + (`test_rearm_restores_a_spec_outside_every_root_it_could_be_confined_to`). """ _resolve_repo(tmp_path) wt = tmp_path / ".bmad-loop" / "runs" / "wt-mount" # the mount, which holds no spec @@ -1759,12 +1860,213 @@ def boom(spec_path, *, confine_root): monkeypatch.setattr(runs.devcontract, "strip_auto_run_result", boom) with pytest.raises(runs.RearmError, match="No space left on device"): - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) assert spec.read_bytes() == before # the undo reached a spec outside the mount assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.ESCALATED +def test_rearm_restores_a_spec_outside_every_root_it_could_be_confined_to(tmp_path, monkeypatch): + """The undo has to reach the spec wherever its three sibling writers reached it. + + An artifacts folder configured OUTSIDE the checkout is supported configuration — + `bmadconfig` resolves one, `verify.spec_within_roots` trusts it, and + `_spec_is_shared_with_the_redrive` treats a spec that lands there as first-class and + reachable by the re-drive. On that shape neither candidate root can confine the path: + the mount cannot, and neither can the project, so `task_spec_root`'s fallback names a + root the spec is lexically outside of. + + `frontmatter.set_frontmatter_status`, `verify.set_frontmatter_field` and + `devcontract._atomic_write_spec` all select their writer on that same lexical test and + simply take the plain no-follow arm, so the flip, the strip and the re-stamp LAND. + `_restore_rearmed_spec` called `atomic_write_bytes_confined` unconditionally, so the + undo alone raised `UnconfinedWriteError` — the transaction's write set going + unhonoured on exactly the specs it was still able to break, and the operator left with + a flipped, stripped spec on a story the run still called ESCALATED plus a second error + masking the first. A writer that refuses where its siblings write is not extra safety. + + The project deliberately sits UNDER `tmp_path` here so the spec can be a sibling of + it: that is the only way to build a path outside both roots without leaving the + fixture's tree. + + The fault is raised from `save_state` rather than from a git probe because it must be + reached unconditionally: `_stale_restore_residue` returns before touching git when the + task carries no restore latch, so a `commits_above` injection would never fire here. + + Ablation: make `_restore_rearmed_spec` call `atomic_write_bytes_confined` + unconditionally again and this reddens on the `match=` — the raise becomes + "cannot restore ... UnconfinedWriteError" instead of the fault the re-arm aborted on + — with the byte comparison reddening behind it. + """ + project = tmp_path / "proj" + project.mkdir() + _resolve_repo(project) + spec = tmp_path / "artifacts" / "spec.md" # outside the project, and outside any mount + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text( + "---\nstatus: blocked\n---\n\n## Intent\n\nx\n\n## Auto Run Result\n\nterminal\n", + encoding="utf-8", + ) + before = spec.read_bytes() + run_dir, _, _ = _escalated_run(project, spec_file=str(spec)) + + def boom(run_dir_, state_): + raise MemoryError("nothing to do with the spec") + + monkeypatch.setattr(runs, "save_state", boom) + + # the flip and the strip both LAND on this path (their writers degrade to the plain + # arm), so there is a real published write for the undo to put back + with pytest.raises(MemoryError, match="nothing to do with the spec"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before + assert load_state(run_dir).tasks["6-4-cli-list-command"].phase == Phase.ESCALATED + (aborted,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-aborted"] + assert aborted["rollback"] == "restored" + + +def test_rearm_reports_a_failed_rollback_through_the_plain_arm(tmp_path, monkeypatch): + """The undo's `failed` outcome has to be reachable through BOTH of its writers. + + `tests/test_runs.py::test_rearm_reports_a_rollback_that_itself_failed_and_keeps_the_original_fault` + injects at `runs.atomic_write_bytes_confined`, and its fixture always puts the spec + under the project, so it only ever grades the CONFINED arm. The plain + `atomic_write_bytes` arm added for the out-of-every-root shape had no `failed` + coverage at all — the sibling row above grades that arm's `"restored"` outcome only, + so a plain arm that raised the wrong type, or swallowed instead of raising, was + invisible. + + Same three claims as the confined row, on the other writer: the `RearmError` names the + spec, the record says `failed`, and the ORIGINAL fault rides in the exception chain + because the restore raises WHILE that fault is being handled. + + Ablation: make `_restore_rearmed_spec` take `atomic_write_bytes_confined` + unconditionally and this reddens on the INJECTED-fault assertion. That ablation is + the one that matters and the one the three claims above cannot catch on their own: + the confined writer refuses this out-of-root path with `UnconfinedWriteError`, which + IS an `OSError`, so it produces the same `RearmError`, the same `failed` record and + the same chained `MemoryError` — every claim stays true while the plain arm this row + exists for is never reached. Naming the injected error is what tells the two apart. + """ + project = tmp_path / "proj" + project.mkdir() + _resolve_repo(project) + spec = tmp_path / "artifacts" / "spec.md" # outside the project, and outside any mount + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text( + "---\nstatus: blocked\n---\n\n## Intent\n\nx\n\n## Auto Run Result\n\nterminal\n", + encoding="utf-8", + ) + run_dir, _, _ = _escalated_run(project, spec_file=str(spec)) + + def boom(run_dir_, state_): + raise MemoryError("nothing to do with the spec") + + def no_space(*_a, **_kw): + raise OSError(28, "No space left on device") + + monkeypatch.setattr(runs, "save_state", boom) + # ONLY the undo's out-of-root writer: the flip and the strip reach this path through + # `verify` and `devcontract`, so this cannot pre-empt the writes it is meant to fail + # to undo + monkeypatch.setattr(runs, "atomic_write_bytes", no_space) + + with pytest.raises(runs.RearmError, match="cannot restore") as excinfo: + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert str(spec) in str(excinfo.value) + # the fault the operator is shown is the one the PLAIN arm raised. Without this the + # row cannot tell its own writer apart from the confined one refusing the same path + assert "No space left on device" in str(excinfo.value) + chain = [] + exc: BaseException | None = excinfo.value + while exc is not None: + chain.append(exc) + exc = exc.__cause__ or exc.__context__ + assert any(isinstance(e, MemoryError) for e in chain) # the original fault survives + (aborted,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-aborted"] + assert aborted["rollback"] == "failed" + assert "MemoryError" in aborted["error"] + + +@pytest.mark.skipif(sys.platform == "win32", reason="POSIX symlinks") +def test_rearm_rollback_replaces_a_link_planted_at_the_spec_rather_than_writing_through_it( + tmp_path, monkeypatch +): + """The out-of-root undo replaces the NAME, so a link planted at it cannot aim the + captured bytes into whatever it points at — on the shape this row drives, where that + file's bytes DIFFER from the preimage. + + That scope is the short-circuit's, not a hedge. `_restore_rearmed_spec` answers + `"unchanged"` and writes NOTHING when `spec_path.read_bytes()` already equals the + preimage, and that read follows the link — so a link aimed at a byte-equal file is + never replaced and there is nothing left for `follow_symlinks` to decide. Reaching + that shape needs a second actor mutating the spec's name mid-window, which this + story's triage log has repeatedly found unreachable while the run is paused and the + resolve session that wrote the spec has terminated. It is therefore left ungraded + rather than pinned by a row built on an actor that does not exist. + + `_restore_rearmed_spec`'s plain arm passes `follow_symlinks=False`, matching the + three writers it undoes (`frontmatter.set_frontmatter_status` states the rule). + That argument was the one thing on this path with no caller-level coverage: the + sibling rows above drive the arm over a plain regular file, where following or not + following resolves to the same inode, so dropping the argument left them green while + the undo silently gained the default's `path.resolve()` — and with it a window in + which the last thing that touches the spec's name decides which file this re-arm's + preimage lands in. + + The window is the widened transaction's own: the flip and the strip publish to the + real file, then the guard's whole residue/advance/`save_state` tail runs before the + undo looks at the name again. This row plants the link at the last moment inside that + tail — from the injected `save_state`, so the redirection is in place before the + rollback and after every write it exists to put back. + + The `restored` record is the third claim rather than a redundant one: the undo has to + read the link (seeing the OTHER file's bytes, which do not match the preimage), take + its writer, and land — the same three steps a silent write-through also takes, which + is why the byte assertions and not the record are what tell the two apart. + + Ablation: drop `follow_symlinks=False` from `_restore_rearmed_spec`'s plain + `atomic_write_bytes` call and this reddens on the FIRST assertion — the preimage + lands in the unrelated file — with `not spec.is_symlink()` reddening behind it. The + final byte comparison stays green through that ablation (it reads THROUGH the link), + so it cannot carry this row on its own. + """ + project = tmp_path / "proj" + project.mkdir() + _resolve_repo(project) + spec = tmp_path / "artifacts" / "spec.md" # outside the project, and outside any mount + spec.parent.mkdir(parents=True, exist_ok=True) + spec.write_text( + "---\nstatus: blocked\n---\n\n## Intent\n\nx\n\n## Auto Run Result\n\nterminal\n", + encoding="utf-8", + ) + before = spec.read_bytes() + bystander = tmp_path / "artifacts" / "someone-elses-notes.md" + bystander.write_bytes(b"not this re-arm's file\n") + bystander_before = bystander.read_bytes() + run_dir, _, _ = _escalated_run(project, spec_file=str(spec)) + + def boom(run_dir_, state_): + # the flip and the strip have already LANDED on the real file; the name is + # redirected here, inside the window, before the undo looks at it again + spec.unlink() + spec.symlink_to(bystander) + raise MemoryError("nothing to do with the spec") + + monkeypatch.setattr(runs, "save_state", boom) + + with pytest.raises(MemoryError, match="nothing to do with the spec"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert bystander.read_bytes() == bystander_before # the preimage did NOT go through + assert not spec.is_symlink() # the name was replaced, whatever it pointed at + assert spec.read_bytes() == before + (aborted,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-aborted"] + assert aborted["rollback"] == "restored" + + def test_rearm_journals_the_spec_baseline_it_overwrote(tmp_path): """A claim the re-stamp normalizes away is the only trace of a divergence the gate can no longer report, so it lands in the journal on the way out — read @@ -1778,7 +2080,7 @@ def test_rearm_journals_the_spec_baseline_it_overwrote(tmp_path): old_head = _resolve_repo(tmp_path) run_dir, _spec, new_head = _escalated_spec_run(tmp_path, old_head) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] assert entry["overwritten"] == old_head @@ -1787,7 +2089,7 @@ def test_rearm_journals_the_spec_baseline_it_overwrote(tmp_path): # a second re-arm has nothing left to overwrite: no duplicate record save_state(run_dir, _rearmable(run_dir)) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert len([e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"]) == 1 @@ -1813,7 +2115,7 @@ def test_rearm_does_not_report_a_divergence_the_run_never_had(tmp_path): old_head = _resolve_repo(tmp_path) run_dir, spec, new_head = _escalated_spec_run(tmp_path, old_head, recorded=old_head) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) # the re-stamp itself ran: this row is about what was REPORTED, not what was skipped assert verify.read_frontmatter(spec)["baseline_revision"] == new_head @@ -1849,7 +2151,7 @@ def test_rearm_reports_a_claim_the_advanced_head_would_have_masked(tmp_path): encoding="utf-8", ) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] assert entry["overwritten"] == new_head # the claim, carried verbatim @@ -1866,7 +2168,7 @@ def test_rearm_prefers_the_fresh_revision_when_the_spec_carries_both_keys(tmp_pa tmp_path, old_head, extra=f"baseline_commit: {'a' * 40}\n" ) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) (entry,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-baseline-restamped"] assert entry["overwritten"] == old_head # NOT the stale baseline_commit @@ -1902,7 +2204,7 @@ def test_build_context_tolerates_non_utf8_present_spec(tmp_path): (stories_dir / f"{key}-slug.md").write_bytes(_BAD_UTF8) # a real spec, undecodable run_dir, state, _ = _escalated_run(tmp_path, source="stories") - path = resolve.build_context(state, run_dir, key, isolation="") # must not raise + path = _context(state, run_dir, key, isolation="") # must not raise ctx = json.loads(path.read_text(encoding="utf-8")) assert ctx["stories"]["spec_folder"] == "" # best-effort context still produced assert "sentinel" not in ctx["stories"] # the undecodable spec yields no sentinel @@ -1917,7 +2219,7 @@ def test_build_context_tolerates_non_utf8_sentinel(tmp_path): (stories_dir / f"{key}-unresolved.md").write_bytes(_BAD_UTF8) # undecodable sentinel run_dir, state, _ = _escalated_run(tmp_path, source="stories", sentinel_kind="unresolved") - path = resolve.build_context(state, run_dir, key, isolation="") # must not raise + path = _context(state, run_dir, key, isolation="") # must not raise ctx = json.loads(path.read_text(encoding="utf-8")) assert ctx["stories"]["sentinel"]["kind"] == "unresolved" assert ctx["stories"]["sentinel"]["blocking_condition"] == "" # unreadable → empty @@ -1938,7 +2240,7 @@ def test_rearm_non_utf8_present_spec_fails_clean_and_stays_armed(tmp_path): run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spec), source="stories") with pytest.raises(runs.RearmError) as exc: - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) assert "UTF-8" in str(exc.value) and "resolve" in str(exc.value) assert spec.read_bytes() == _BAD_UTF8 # spec untouched task = load_state(run_dir).tasks[key] @@ -1958,7 +2260,9 @@ def test_rearm_tolerates_non_utf8_sentinel(tmp_path): tmp_path, spec_file=str(sentinel), source="stories", sentinel_kind="unresolved" ) - assert runs.rearm_escalation(run_dir, isolated_redrive=False) == key # must not raise + assert ( + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) == key + ) # must not raise assert not sentinel.exists() # cleared by deletion assert (run_dir / "sentinels" / f"{key}-unresolved.md").is_file() # copy preserved assert load_state(run_dir).tasks[key].spec_file is None # cleared → PENDING re-dispatch @@ -1991,7 +2295,7 @@ def test_rearm_rejects_non_escalation_stage(tmp_path): ), ) with pytest.raises(runs.RearmError, match="not paused at an escalation"): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) def test_rearm_rejects_unescalated_story(tmp_path): @@ -1999,7 +2303,7 @@ def test_rearm_rejects_unescalated_story(tmp_path): task.phase = Phase.DONE # terminal but not escalated save_state(run_dir, state) with pytest.raises(runs.RearmError, match="not escalated"): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) # ------------------------------------------------- _gather_escalations @@ -2007,12 +2311,12 @@ def test_rearm_rejects_unescalated_story(tmp_path): def test_gather_escalations_reads_one_escalation_once_per_distinct_id(tmp_path): """The outermost surface DW-7 names. `_gather_escalations` walks the append-only - `task.sessions` — which a re-arm deliberately does NOT clear — and reads - `tasks//escalation.json` once per record. While the ESCALATED - restart re-minted an id byte-equal to the abandoned attempt's, BOTH records - addressed the one file, so the abandoned cycle's escalation was returned a second - time as the fresh session's. Bumping `generation` gives the fresh record its own - id, and the file is read exactly once.""" + `task.sessions` — which a re-arm deliberately does NOT clear — and now opens each + distinct `tasks/` directory once. Before that reader guard, an + ESCALATED restart that re-minted the abandoned attempt's id made BOTH records + address one file and returned its escalation twice. Bumping `generation` gives the + fresh record its own artifact namespace; the reader also degrades safely on older + persisted state where the collision already exists.""" run_dir, state, task = _escalated_run(tmp_path) key = "6-4-cli-list-command" abandoned = _session_task_id(key, "triage", 1, 0) @@ -2029,14 +2333,1029 @@ def test_gather_escalations_reads_one_escalation_once_per_distinct_id(tmp_path): encoding="utf-8", ) - found = resolve._gather_escalations(run_dir, state, key) + found, _ = resolve._gather_escalations(run_dir, state, key) assert [e["detail"] for e in found] == ["abandoned cycle"] # once, not twice - # the pre-fix shape for contrast: one shared id makes the SAME file answer both - # records, and the abandoned escalation is attributed to the fresh session too + # DW-71: the id bump only protects records minted AFTER it. State persisted + # before the bump still carries two records under ONE id, both addressing that + # directory's single mutable escalation.json — the reader itself has to return + # the escalation once rather than attribute it to the fresh session too. task.sessions[1] = SessionRecord(task_id=abandoned, role="dev", status="completed") - collided = resolve._gather_escalations(run_dir, state, key) - assert [e["detail"] for e in collided] == ["abandoned cycle", "abandoned cycle"] + collided, _ = resolve._gather_escalations(run_dir, state, key) + assert [e["detail"] for e in collided] == ["abandoned cycle"] + + +def test_gather_escalations_opens_a_repeated_task_id_once(tmp_path, monkeypatch): + """DW-71's own leg, watched at the I/O rather than the return value. + + Content de-duplication would hide a re-read behind the identical entry it + yields, so "returned once" alone cannot tell the `seen_ids` guard from the + content map. Two records under one `task_id` must OPEN that directory's + artifacts exactly once — which is also what stops a directory rewritten + mid-pass from answering two records differently.""" + run_dir, state, task = _escalated_run(tmp_path) + key = "6-4-cli-list-command" + shared = _session_task_id(key, "triage", 1, 0) + task.sessions.clear() + for _ in range(2): + task.sessions.append(SessionRecord(task_id=shared, role="dev", status="completed")) + esc_dir = run_dir / "tasks" / shared + esc_dir.mkdir(parents=True, exist_ok=True) + result_file = esc_dir / "result.json" + result_file.write_text(json.dumps({"escalations": []}), encoding="utf-8") + esc_file = esc_dir / "escalation.json" + esc_file.write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "shared id"}]}), + encoding="utf-8", + ) + + reads: list[str] = [] + real_read_text = Path.read_text + + def counting_read_text(self, *args, **kwargs): + reads.append(str(self)) + return real_read_text(self, *args, **kwargs) + + # `monkeypatch.context()`, NOT a bare `setattr` + `undo()`: the autouse + # `_isolate_state_root` / `_isolate_mux_registry` fixtures record onto the SAME + # function-scoped monkeypatch instance this test receives (conftest says so in + # `_isolate_state_root`'s own docstring), so an explicit `undo()` here would roll + # back the suite's `BMAD_LOOP_STATE_DIR` isolation too, mid-test. + with monkeypatch.context() as mp: + mp.setattr(Path, "read_text", counting_read_text) + found, _ = resolve._gather_escalations(run_dir, state, key) + + assert reads.count(str(result_file)) == 1 # each artifact once, not once per record + assert reads.count(str(esc_file)) == 1 + assert [e["detail"] for e in found] == ["shared id"] + + +def _two_session_dirs(tmp_path): + """A task carrying TWO records with DISTINCT `task_id`s, plus both task + directories. `task.sessions` is append-only and chronological, so `sessions[1]` + is the NEWER attempt and `reversed(...)` must reach its directory first. + + This shape exists because no single-directory row can see either of this + reader's cross-session contracts: rescope the content map per directory, or + drop `reversed`, and every one-directory row below stays green.""" + run_dir, state, task = _escalated_run(tmp_path) + key = "6-4-cli-list-command" + older = _session_task_id(key, "triage", 1, 0) + newer = _session_task_id(key, "triage", 1, 1) # post-bump: the -g1 namespace + assert older != newer + task.sessions.clear() + dirs: list[Path] = [] + for task_id in (older, newer): + task.sessions.append(SessionRecord(task_id=task_id, role="dev", status="completed")) + d = run_dir / "tasks" / task_id + d.mkdir(parents=True, exist_ok=True) + dirs.append(d) + return run_dir, state, key, dirs[0], dirs[1] + + +def test_gather_escalations_dedupes_one_entry_across_two_sessions(tmp_path): + """De-duplication is GLOBAL across the pass, not scoped to one directory. + + An escalation a retry does not resolve is re-raised by the next attempt, so two + DIFFERENT `tasks//` directories carry the byte-identical entry and the + operator learns nothing from the repeat. This is the only row that can tell a + global content map from a per-directory one.""" + run_dir, state, key, older_dir, newer_dir = _two_session_dirs(tmp_path) + entry = {"type": "spec-gap", "severity": "CRITICAL", "detail": "unresolved across attempts"} + for d in (older_dir, newer_dir): + (d / "escalation.json").write_text(json.dumps({"escalations": [entry]}), encoding="utf-8") + + found, _ = resolve._gather_escalations(run_dir, state, key) + assert [e["detail"] for e in found] == ["unresolved across attempts"] + + +def test_gather_escalations_orders_distinct_sessions_newest_first(tmp_path): + """The documented "newest first" order is a CROSS-SESSION property: nothing + inside one directory can pin it, because `reversed(task.sessions)` is what + reaches the newer record's directory before the older one's. Drop `reversed` + and only this row notices.""" + run_dir, state, key, older_dir, newer_dir = _two_session_dirs(tmp_path) + for d, detail in ((older_dir, "older"), (newer_dir, "newer")): + (d / "escalation.json").write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": detail}]}), + encoding="utf-8", + ) + + found, _ = resolve._gather_escalations(run_dir, state, key) + assert [e["detail"] for e in found] == ["newer", "older"] + + +def _task_dir(run_dir, task): + """Where `_gather_escalations` looks for result.json / escalation.json, DERIVED + from the session record the fixture actually appended — never a literal. + + A hardcoded directory name is a false green waiting on a fixture change: it can + drift off the record the reader walks, and a row asserting an EMPTY result would + then pass because nothing was read rather than because the filter worked.""" + d = run_dir / "tasks" / task.sessions[-1].task_id + d.mkdir(parents=True, exist_ok=True) + return d + + +def test_gather_escalations_returns_a_mirrored_entry_once(tmp_path): + """DW-68/72. The sweep skill's contract (bmad-loop-sweep/automation-mode.md) + tells a producer to write escalation.json and then mirror the same entries into + result.json `escalations` — so every COMPLIANT escalation reached the operator + twice. The mirroring stays; the reader absorbs it. Asserted through + `build_context` because `context.json` is the surface the human reads.""" + run_dir, state, task = _escalated_run(tmp_path) + entry = {"type": "spec-gap", "severity": "CRITICAL", "detail": "mirrored once"} + # Same JSON object, deliberately authored in a different member order. Raw + # `json.dumps(esc)` keys would treat these as distinct; `sort_keys=True` must + # make the de-duplication key semantic rather than source-order-sensitive. + reordered = {"detail": "mirrored once", "severity": "CRITICAL", "type": "spec-gap"} + task_dir = _task_dir(run_dir, task) + for fname, value in (("result.json", entry), ("escalation.json", reordered)): + (task_dir / fname).write_text(json.dumps({"escalations": [value]}), encoding="utf-8") + + ctx = json.loads( + _context(state, run_dir, "6-4-cli-list-command", isolation="").read_text(encoding="utf-8") + ) + assert ctx["escalations"] == [entry] + + +def test_gather_escalations_keeps_distinct_entries_from_both_files(tmp_path): + """De-duplication removes only the exact repeat. A directory whose result.json + carries A and whose escalation.json carries A + B still yields both, in + newest-first order (result.json before escalation.json) — the guard must not + collapse a partially-mirrored pair into one.""" + run_dir, state, task = _escalated_run(tmp_path) + a = {"type": "spec-gap", "severity": "CRITICAL", "detail": "A"} + b = {"type": "spec-gap", "severity": "CRITICAL", "detail": "B"} + task_dir = _task_dir(run_dir, task) + (task_dir / "result.json").write_text(json.dumps({"escalations": [a]}), encoding="utf-8") + (task_dir / "escalation.json").write_text(json.dumps({"escalations": [a, b]}), encoding="utf-8") + + found, _ = resolve._gather_escalations(run_dir, state, "6-4-cli-list-command") + assert [e["detail"] for e in found] == ["A", "B"] + + +def test_gather_escalations_keeps_full_objects_that_share_a_detail(tmp_path): + """Exact content, not one convenient field, defines a duplicate. Two + escalations may explain the same symptom while identifying different gaps; + both complete dictionaries must reach the resolver.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + first = { + "type": "spec-gap", + "severity": "CRITICAL", + "detail": "same operator-facing explanation", + "location": "SPEC.md", + } + second = { + "type": "environment-gap", + "severity": "CRITICAL", + "detail": "same operator-facing explanation", + "location": "policy.toml", + } + (task_dir / "result.json").write_text( + json.dumps({"escalations": [first, second]}), encoding="utf-8" + ) + + assert resolve._gather_escalations(run_dir, state, "6-4-cli-list-command") == ( + [first, second], + 0, + ) + + +def test_gather_escalations_preserves_result_before_escalation_file_order(tmp_path): + """Within one session directory, result.json precedes escalation.json.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + first = {"severity": "CRITICAL", "detail": "from result"} + second = {"severity": "CRITICAL", "detail": "from escalation"} + (task_dir / "result.json").write_text(json.dumps({"escalations": [first]}), encoding="utf-8") + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [second]}), encoding="utf-8" + ) + + assert resolve._gather_escalations(run_dir, state, "6-4-cli-list-command") == ( + [first, second], + 0, + ) + + +def test_gather_escalations_keeps_a_duplicates_first_position(tmp_path): + """A later copy must not move an entry behind intervening distinct content.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + first = {"severity": "CRITICAL", "detail": "first"} + second = {"severity": "CRITICAL", "detail": "second"} + (task_dir / "result.json").write_text(json.dumps({"escalations": [first]}), encoding="utf-8") + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [second, first]}), encoding="utf-8" + ) + + assert resolve._gather_escalations(run_dir, state, "6-4-cli-list-command") == ( + [first, second], + 0, + ) + + +def test_gather_escalations_dedupes_repeats_inside_one_list(tmp_path): + """The content map spans the whole pass, including one producer's list.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + entry = {"severity": "CRITICAL", "detail": "listed twice"} + (task_dir / "result.json").write_text( + json.dumps({"escalations": [entry, entry]}), encoding="utf-8" + ) + + assert resolve._gather_escalations(run_dir, state, "6-4-cli-list-command") == ([entry], 0) + + +def test_gather_escalations_keeps_mixed_case_critical_and_drops_non_dicts(tmp_path): + """Delegating the filter preserves its case-insensitive and shape semantics.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + critical = {"severity": "critical", "detail": "case folded"} + preference = {"severity": "PREFERENCE", "detail": "not critical"} + (task_dir / "result.json").write_text( + json.dumps({"escalations": [None, "junk", preference, critical]}), encoding="utf-8" + ) + + assert resolve._gather_escalations(run_dir, state, "6-4-cli-list-command") == ([critical], 0) + + +def test_gather_escalations_skips_a_non_utf8_artifact(tmp_path): + """DW-70/73. `UnicodeDecodeError` is a `ValueError`, not an `OSError`, so the + old `except (OSError, json.JSONDecodeError)` let a non-UTF-8 artifact crash + `build_context` — the interactive resolve path, an OBSERVATION surface that must + degrade. The bad file costs its own contents and nothing more.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + (task_dir / "result.json").write_bytes(_BAD_UTF8) + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "still readable"}]}), + encoding="utf-8", + ) + + ctx = json.loads( + _context(state, run_dir, "6-4-cli-list-command", isolation="").read_text(encoding="utf-8") + ) + assert [e["detail"] for e in ctx["escalations"]] == ["still readable"] + + +def test_gather_escalations_skips_a_plain_json_value_error(tmp_path, monkeypatch): + """`json.loads` raises plain ValueError, not JSONDecodeError, when an integer + exceeds Python's configured digit limit. That malformed file costs only its + contents; its valid sibling still reaches context.json.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + marker = '"detail":' + ("9" * 5000) + (task_dir / "result.json").write_text( + '{"escalations":[{"severity":"CRITICAL",' + marker + "}]}", encoding="utf-8" + ) + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "sibling survives"}]}), + encoding="utf-8", + ) + + real_loads = json.loads + + def loads_with_digit_limit(data, *args, **kwargs): + if marker in data: + raise ValueError("integer exceeds configured digit limit") + return real_loads(data, *args, **kwargs) + + with monkeypatch.context() as mp: + mp.setattr(resolve.json, "loads", loads_with_digit_limit) + path = _context(state, run_dir, "6-4-cli-list-command", isolation="") + + ctx = json.loads(path.read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == ["sibling survives"] + + +def test_gather_escalations_skips_a_json_recursion_error(tmp_path): + """A deeply nested artifact can exceed the decoder's recursion guard. + + Confirm the real decoder failure first so this stays a regression test for + ``RecursionError`` rather than another synthetic exception row. The bad file + still costs only its own contents; its valid sibling reaches context.json. + """ + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + nested = json_recursion_payload() + malformed = '{"escalations":[{"severity":"CRITICAL","detail":' + nested + "}]}" + with pytest.raises(RecursionError): + json.loads(malformed) + (task_dir / "result.json").write_text(malformed, encoding="utf-8") + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "sibling survives"}]}), + encoding="utf-8", + ) + + ctx = json.loads( + _context(state, run_dir, "6-4-cli-list-command", isolation="").read_text(encoding="utf-8") + ) + assert [e["detail"] for e in ctx["escalations"]] == ["sibling survives"] + + +def test_gather_escalations_skips_a_canonicalization_recursion_error(tmp_path, monkeypatch): + """Canonical-key construction is part of the guarded artifact read too.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + bad = {"severity": "CRITICAL", "detail": "canonicalization recurses"} + sibling = {"severity": "CRITICAL", "detail": "sibling survives"} + (task_dir / "result.json").write_text(json.dumps({"escalations": [bad]}), encoding="utf-8") + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [sibling]}), encoding="utf-8" + ) + real_dumps = json.dumps + + def dumps_with_recursion_error(value, *args, **kwargs): + if value == bad: + raise RecursionError("canonicalization depth exceeded") + return real_dumps(value, *args, **kwargs) + + with monkeypatch.context() as mp: + mp.setattr(resolve.json, "dumps", dumps_with_recursion_error) + path = _context(state, run_dir, "6-4-cli-list-command", isolation="") + + ctx = json.loads(path.read_text(encoding="utf-8")) + assert ctx["escalations"] == [sibling] + + +@pytest.mark.parametrize("bad", [None, 1, "x", {}]) +def test_gather_escalations_skips_a_non_list_escalations_field(tmp_path, monkeypatch, bad): + """DW-70/73's other half. `escalation.critical_escalations` iterates + `escalations` with no list guard of its own, so `{"escalations": null}` raised + `TypeError` straight out of `build_context`. The guard sits in this caller; the + shared predicate stays the single definition of CRITICAL. + + Every parameter must fail when the list guard is ablated. ``None`` and ``1`` + raise without it; the call trace below distinguishes the iterable ``"x"`` and + ``{}`` shapes, which the shared filter would otherwise accept as empty.""" + run_dir, state, task = _escalated_run(tmp_path) + task_dir = _task_dir(run_dir, task) + (task_dir / "result.json").write_text(json.dumps({"escalations": bad}), encoding="utf-8") + (task_dir / "escalation.json").write_text( + json.dumps({"escalations": [{"severity": "CRITICAL", "detail": "sibling survives"}]}), + encoding="utf-8", + ) + + filtered: list[dict] = [] + real_critical_escalations = resolve.critical_escalations + + def recording_critical_escalations(doc): + filtered.append(doc) + return real_critical_escalations(doc) + + with monkeypatch.context() as mp: + mp.setattr(resolve, "critical_escalations", recording_critical_escalations) + path = _context(state, run_dir, "6-4-cli-list-command", isolation="") + + ctx = json.loads(path.read_text(encoding="utf-8")) + assert filtered == [ + { + "escalations": [ + {"severity": "CRITICAL", "detail": "sibling survives"}, + ] + } + ] + assert [e["detail"] for e in ctx["escalations"]] == ["sibling survives"] + + +def test_gather_escalations_preference_only_yields_nothing(tmp_path): + """The CRITICAL-only filter is unchanged by the de-duplication rewrite: a + directory carrying only non-CRITICAL entries contributes nothing, and mirroring + a PREFERENCE across both files still contributes nothing. + + The second half is the POSITIVE CONTROL, and it is what makes the first half + mean anything. `== []` passes just as well when the directory was never read, so + the same files are re-written with a CRITICAL alongside the PREFERENCE and that + entry must come back. Absence then evidences the severity filter rather than an + unread path.""" + run_dir, state, task = _escalated_run(tmp_path) + key = "6-4-cli-list-command" + pref = {"type": "nit", "severity": "PREFERENCE", "detail": "ignore me"} + task_dir = _task_dir(run_dir, task) + for fname in ("result.json", "escalation.json"): + (task_dir / fname).write_text(json.dumps({"escalations": [pref]}), encoding="utf-8") + + assert resolve._gather_escalations(run_dir, state, key) == ([], 0) + + crit = {"type": "spec-gap", "severity": "CRITICAL", "detail": "kept"} + for fname in ("result.json", "escalation.json"): + (task_dir / fname).write_text(json.dumps({"escalations": [pref, crit]}), encoding="utf-8") + found, _ = resolve._gather_escalations(run_dir, state, key) + assert [e["detail"] for e in found] == ["kept"] # this directory IS read + + +# -------------------------------------- DW-11: the escalation watermark + + +def _watermarked_trail(tmp_path, per_session): + """A task whose append-only `sessions` list carries ONE record per element of + `per_session`, each with its own `tasks//escalation.json` holding that + record's CRITICAL details. Returns `(run_dir, state, task, key)` with the state + already saved, so a row can re-arm it without re-saving by hand. + + The ids are minted through `engine._session_task_id`, varying the SEQ inside + generation 0 — the trail one pre-re-arm cycle leaves behind. Distinctness is + asserted rather than assumed: a shared id collapses into the reader's `seen_ids` + guard, leaving one directory and one side to route to, and every row below would + then pass with the filter ablated. Varying the seq (not the generation) also + keeps the whole namespace clear of the ids a LATER re-arm mints, so a re-drive + record cannot silently overwrite a trail artifact. + """ + run_dir, state, task = _escalated_run(tmp_path) + key = "6-4-cli-list-command" + task.sessions.clear() + for seq, details in enumerate(per_session, start=1): + task_id = _session_task_id(key, "review", seq, 0) + assert task_id not in {r.task_id for r in task.sessions} + task.sessions.append(SessionRecord(task_id=task_id, role="dev", status="completed")) + d = run_dir / "tasks" / task_id + d.mkdir(parents=True, exist_ok=True) + (d / "escalation.json").write_text( + json.dumps( + { + "escalations": [ + {"type": "spec-gap", "severity": "CRITICAL", "detail": detail} + for detail in details + ] + } + ), + encoding="utf-8", + ) + save_state(run_dir, state) + return run_dir, state, task, key + + +def _redrive_escalates(run_dir, key, detail, *, escalated=False): + """Append the record + artifact a re-driven session that escalated again leaves + behind — through `record_session`, the SOLE mutation of `task.sessions` in + `src/`, which is what makes a length watermark meaningful. The id carries the + re-arm's own generation, exactly as `engine._session_task_id` would mint it.""" + state = load_state(run_dir) + task = state.tasks[key] + assert task.generation > 0 # a re-arm ran, so this id is in a fresh namespace + task_id = _session_task_id(key, "review", 1, task.generation) + assert task_id not in {r.task_id for r in task.sessions} + task.record_session(SessionRecord(task_id=task_id, role="dev", status="completed")) + d = run_dir / "tasks" / task_id + d.mkdir(parents=True, exist_ok=True) + (d / "escalation.json").write_text( + json.dumps( + {"escalations": [{"type": "spec-gap", "severity": "CRITICAL", "detail": detail}]} + ), + encoding="utf-8", + ) + if escalated: + task.phase = Phase.ESCALATED + save_state(run_dir, state) + + +def test_gather_escalations_shows_the_whole_trail_at_watermark_zero(tmp_path): + """The default is the PRE-DW-11 walk, byte-for-byte. 0 is what a task that was + never resolved carries and what a pre-upgrade `state.json` deserializes to, so + this row is also the legacy-state contract at the reader.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["older"], ["newer"]]) + assert task.escalations_resolved_upto == 0 + + found, withheld = resolve._gather_escalations(run_dir, state, key) + assert [e["detail"] for e in found] == ["newer", "older"] + assert withheld == 0 + + +def test_gather_escalations_hides_sessions_below_the_watermark(tmp_path): + """The defect DW-11 names. `task.sessions` is append-only and a re-arm + deliberately does not clear it, so a second resolve cycle re-presented every + escalation the story ever raised — interleaved with the new ones and with + nothing marking which was which, against a skill contract that is singular + ("present THE escalation"). + + Ablation: ignore `start` in `_gather_escalations` (route everything to `found`) + and this row fails by showing the answered entry again.""" + run_dir, state, _task, key = _watermarked_trail( + tmp_path, [["answered last cycle"], ["raised since"]] + ) + + found, withheld = resolve._gather_escalations(run_dir, state, key, start=1) + assert [e["detail"] for e in found] == ["raised since"] + assert withheld == 1 + + +def test_gather_escalations_counts_the_entries_it_withheld(tmp_path): + """The number the operator is shown is the count of DISTINCT withheld entries, + not of sessions or of directories — and it comes from the same single walk that + produced the shown list, never a second call subtracting lengths.""" + run_dir, state, _task, key = _watermarked_trail(tmp_path, [["a", "b", "c"], ["new"]]) + + found, withheld = resolve._gather_escalations(run_dir, state, key, start=1) + assert [e["detail"] for e in found] == ["new"] + assert withheld == 3 + + +def test_gather_escalations_does_not_count_an_entry_it_still_shows(tmp_path): + """ "Not shown" is the claim the number makes, so it must never count something + the operator can see. An escalation the re-drive re-raised appears on BOTH sides + of the watermark: it is shown once (the newest-first content map) and contributes + 0 to the count, while its answered-only sibling contributes 1. + + The sibling is the in-row positive control: an `assert withheld == 0` alone would + pass just as well if the answered directory were never read at all. + + Ablation: drop the `key not in found` clause from the count and this reddens at + 2 != 1.""" + run_dir, state, _task, key = _watermarked_trail( + tmp_path, + [["re-raised by the re-drive", "answered and gone"], ["re-raised by the re-drive"]], + ) + + found, withheld = resolve._gather_escalations(run_dir, state, key, start=1) + assert [e["detail"] for e in found] == ["re-raised by the re-drive"] # once, not twice + assert withheld == 1 # "answered and gone" only + + +def test_gather_escalations_attributes_a_task_id_spanning_the_watermark_to_the_shown_side( + tmp_path, +): + """One `task_id` on an answered record AND an unanswered one — the shape the + pre-`generation` id namespace produced, which persisted state still carries. The + `seen_ids` guard opens that directory ONCE, at its newest occurrence, which is + the unanswered side: the entry is SHOWN. Over-showing is the conservative + direction; the alternative buries an escalation on an ambiguity. + + Ablation: walk the trail FORWARD — `for index, session in enumerate(task.sessions)` + with `target = found if index >= start else answered`, a rewrite that still reads + correct and leaves every other row in this block green except the ordering sibling + — and this reddens at `([], 1)`. The shared directory is then opened at its + ANSWERED occurrence, so the escalation is buried AND counted as already answered: + the second member is what catches that, which is why the assertion is a tuple and + not the shown list alone. MEASURED, and the recipe is specific for a reason: + deleting the `seen_ids` guard does NOT redden this row (the directory is read + twice, but the key lands in `found` first and the count's `key not in found` + clause absorbs the duplicate), so `seen_ids` is graded by its own siblings above, + not here.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["spans the watermark"]]) + shared = task.sessions[0].task_id + task.sessions.append(SessionRecord(task_id=shared, role="dev", status="completed")) + save_state(run_dir, state) + + assert resolve._gather_escalations(run_dir, state, key, start=1) == ( + [{"type": "spec-gap", "severity": "CRITICAL", "detail": "spans the watermark"}], + 0, + ) + + +def test_gather_escalations_with_no_sessions_is_empty_and_reports_nothing(tmp_path): + run_dir, state, task, key = _watermarked_trail(tmp_path, []) + assert task.sessions == [] + assert resolve._gather_escalations(run_dir, state, key) == ([], 0) + + +def test_gather_escalations_past_the_end_of_the_trail_never_raises(tmp_path): + """A watermark beyond the list — hand-edited state, or a trail that shrank — + must yield an empty shown list, not an IndexError. `start` only SELECTS a map; + nothing is indexed with it, which is what makes that true structurally. + + The `2` is load-bearing: `== ([], 2)` proves both directories were READ and + filtered. An `== []` alone would pass equally if the walk had found nothing.""" + run_dir, state, _task, key = _watermarked_trail(tmp_path, [["first"], ["second"]]) + + assert resolve._gather_escalations(run_dir, state, key, start=9) == ([], 2) + + +def test_rearm_stamps_the_watermark_when_a_resolution_was_recorded(tmp_path): + """The stamp records how much of the audit trail the accepted resolution covered + — a LENGTH of `task.sessions`, taken before the re-drive appends anything. + + Ablation: drop the stamp from `rearm_escalation` and this reddens at 0 != 1, + taking the second-cycle rows below with it.""" + run_dir, _, _ = _escalated_run(tmp_path) + before = load_state(run_dir).tasks["6-4-cli-list-command"] + assert before.escalations_resolved_upto == 0 and len(before.sessions) == 1 + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + task = load_state(run_dir).tasks["6-4-cli-list-command"] + assert task.escalations_resolved_upto == 1 + assert len(task.sessions) == 1 # the trail the watermark indexes still stands + assert task.generation == 1 # positive control: the bump ran on this gesture too + + +def test_rearm_leaves_the_watermark_where_it_was_when_nothing_was_recorded(tmp_path): + """`cmd_resolve` prints "no resolution recorded" and FALLS THROUGH to re-arm, and + both non-interactive re-arm gestures run no session at all. None of them accepted + anything, so none may advance the watermark: escalations no human answered would + otherwise become invisible to every later cycle and be reported as already + answered — the inverse of the defect. + + The generation assertion is the positive control and the discriminator: the bump + is UNCONDITIONAL (it answers session-id reuse, #705, which an abandoned attempt + needs just as much), so this row cannot pass by the re-arm having done nothing. + + Ablation: remove the `if resolution_recorded:` gate and this reddens at 1 != 0.""" + run_dir, _, _ = _escalated_run(tmp_path) + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=False) + + task = load_state(run_dir).tasks["6-4-cli-list-command"] + assert task.escalations_resolved_upto == 0 + assert task.generation == 1 + + +def test_a_second_resolve_cycle_shows_only_what_the_redrive_raised(tmp_path): + """The whole chain with no seam hand-set: escalate, re-arm on a recorded + resolution, let the re-drive append its own session record and artifact, then + build the context a second time. `build_context` reads the watermark off the task + it loaded — nothing in this row passes `start`.""" + run_dir, _state, _task, key = _watermarked_trail(tmp_path, [["the first cycle answered this"]]) + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + _redrive_escalates(run_dir, key, "raised by the re-drive") + + path, withheld, _unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + ctx = json.loads(path.read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == ["raised by the re-drive"] + assert withheld == 1 + + +def test_a_third_cycle_stamps_again_over_the_second(tmp_path): + """TWO accepted cycles in sequence. Every other multi-cycle row stops after one + accepted cycle (`..._shows_only_what_the_redrive_raised`) or pairs an accepted one + with a declining one (`..._over_a_surviving_marker_...`), so nothing pinned that the + watermark keeps ADVANCING. A stamp that fires once and then sticks passes both of + those rows and re-presents cycle 2's answered escalation to every later cycle — + DW-11 itself, surviving one cycle further along. + + Ablation: make the stamp `max(task.escalations_resolved_upto, 1)` and this row + reddens on the third cycle's shown list and its count, while both existing + multi-cycle rows stay green.""" + run_dir, _state, _task, key = _watermarked_trail(tmp_path, [["answered in cycle 1"]]) + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + assert load_state(run_dir).tasks[key].escalations_resolved_upto == 1 + _redrive_escalates(run_dir, key, "answered in cycle 2", escalated=True) + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + task = load_state(run_dir).tasks[key] + assert task.escalations_resolved_upto == 2 # ADVANCED again, over cycle 2's record + assert task.generation == 2 # positive control: both gestures re-armed + + _redrive_escalates(run_dir, key, "raised after cycle 2") + + path, withheld, _unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + ctx = json.loads(path.read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == ["raised after cycle 2"] + assert withheld == 2 # each answered cycle counted once + + +def test_a_rearm_over_a_surviving_marker_does_not_move_the_watermark(tmp_path): + """`resolution.json` SURVIVES the re-arm that consumed it: the only unlink in + `src/` is in `resolve.run_session`, which two of the three re-arm callers never + reach, and nothing deletes it at or after a re-arm. So a marker-presence gate + reads the PREVIOUS cycle's marker as this gesture's own, and a second re-arm + running no session would stamp over an escalation nobody has seen — hiding it + forever and reporting it as already answered. + + The marker is deliberately left on disk here and never removed, which is the + state a real second gesture opens on. + + Ablation: replace the `resolution_recorded` parameter with a + `resolution_path(run_dir, key).is_file()` read inside `rearm_escalation` and this + row reddens twice — the watermark advances to 2, and the context comes back + empty with the new escalation counted as withheld.""" + run_dir, _state, _task, key = _watermarked_trail(tmp_path, [["answered in cycle 1"]]) + + marker = resolve.resolution_path(run_dir, key) + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + assert load_state(run_dir).tasks[key].escalations_resolved_upto == 1 + assert marker.is_file() # MEASURED: nothing deletes it at re-arm + + _redrive_escalates(run_dir, key, "raised after cycle 1", escalated=True) + + # the `--no-interactive` / TUI gesture: no session ran, so nothing was accepted + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=False) + + task = load_state(run_dir).tasks[key] + assert task.escalations_resolved_upto == 1 # NOT len(sessions) == 2 + assert task.generation == 2 # positive control: this gesture DID re-arm + path, withheld, _unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + ctx = json.loads(path.read_text(encoding="utf-8")) + assert [e["detail"] for e in ctx["escalations"]] == ["raised after cycle 1"] + assert withheld == 1 + + +def test_build_context_keeps_the_withheld_count_out_of_the_payload(tmp_path): + """The count is the OPERATOR's, not the agent's: `bmad-loop-resolve/SKILL.md` + documents `escalations` as the list to resolve, and a number for entries the + session cannot see is nothing it can act on. Any spelling of a leak reddens this, + because the key set is compared whole rather than probed for one name.""" + run_dir, _state, _task, key = _watermarked_trail(tmp_path, [["answered"], ["new"]]) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + _redrive_escalates(run_dir, key, "new one") + + path, withheld, _unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + assert withheld == 2 # the count exists... + ctx = json.loads(path.read_text(encoding="utf-8")) + assert set(ctx) == { + "story_key", + "run_id", + "spec_file", + "baseline_commit", + "paused_reason", + "escalations", + "resolution_path", + "restore_supported", + "spec_reaches_the_redrive", + "redrive_base_ref", + } # ...and reaches no field of the agent contract + + +def _unreadable_artifact(run_dir, task, index): + """Corrupt the escalation.json belonging to `task.sessions[index]`. + + Non-UTF-8 bytes rather than bad JSON, so the read fails in + `Path.read_text` — the OSError/UnicodeDecodeError arm of the guard, which is the + shape a transient fault on a network mount actually takes. Returns the path so a + row can assert on the exact member of the sink.""" + fpath = run_dir / "tasks" / task.sessions[index].task_id / "escalation.json" + fpath.write_bytes(b'{"escalations": [\xff\xfe]}') + return fpath + + +def test_gather_escalations_records_an_unreadable_shown_side_artifact(tmp_path): + """The skip the reader has always performed, now SAID. `build_context` is an + observation path and must not raise on a malformed artifact — but the caller then + stamps `escalations_resolved_upto = len(task.sessions)`, which covers the session + that artifact belongs to. Without this signal a transient read fault buries every + escalation in it forever: the next cycle reads the file fine and withholds it as + already answered. + + The readable sibling is the positive control — the degrade still costs exactly its + own artifact's contents and nothing more. + + Ablation: drop the `skipped.add(...)` from the `except` arm and this reddens on an + empty sink while the shown list stays correct.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["older"], ["newer"]]) + corrupt = _unreadable_artifact(run_dir, task, 1) + + skipped: set[str] = set() + found, withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + + assert skipped == {str(corrupt)} + assert [e["detail"] for e in found] == ["older"] # the sibling still read + assert withheld == 0 + + +def test_gather_escalations_ignores_a_skip_below_the_watermark(tmp_path): + """A record the PREVIOUS cycle's watermark already covers cannot be newly buried + by this one, so an unreadable artifact there is not a skip. Recording it would + withhold coverage on every later cycle for a session already answered — a + permanent refusal to advance, which is DW-11 back in the other direction. + + The shown-side escalation is the positive control: the walk ran and routed both + sides. The withheld count legitimately drops to 0 — that number is an operator + advisory and claims nothing durable, which is exactly why the sink is narrower. + + Ablation: drop the `target is found` half of the guard and this reddens on a + one-member sink.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["answered"], ["unanswered"]]) + corrupt = _unreadable_artifact(run_dir, task, 0) + + skipped: set[str] = set() + found, _withheld = resolve._gather_escalations(run_dir, state, key, start=1, skipped=skipped) + + assert skipped == set() + assert corrupt.is_file() # MEASURED: the walk really did reach a corrupt file + assert [e["detail"] for e in found] == ["unanswered"] + + +def test_gather_escalations_does_not_call_an_escalation_less_artifact_a_skip(tmp_path): + """The ordinary shape of a clean `result.json` — no `escalations` key at all — is + not a malformed artifact, and calling it one would withhold coverage from EVERY + resolve cycle on every run, permanently disabling the watermark. The over-signal + is the more dangerous failure of the two, and nothing else grades it: every other + row here seeds an artifact that does carry escalations. + + Ablation: treat a missing `escalations` key as malformed (delete the + `if "escalations" not in doc: continue` arm) and this reddens with the + `result.json` in the sink.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["raised"]]) + (run_dir / "tasks" / task.sessions[0].task_id / "result.json").write_text( + json.dumps({"status": "completed"}), encoding="utf-8" + ) + + skipped: set[str] = set() + found, _withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + + assert skipped == set() + assert [e["detail"] for e in found] == ["raised"] + + +def test_gather_escalations_records_a_non_list_escalations_field(tmp_path): + """`{"escalations": null}` is the shape the reader's `list` check exists for, and + it is malformed rather than absent: something wrote the key, and what it holds + cannot be shown. The sibling row above draws the other side of that line. + + Ablation: fold the non-list arm back into a bare `continue` and this reddens.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["raised"]]) + fpath = run_dir / "tasks" / task.sessions[0].task_id / "escalation.json" + fpath.write_text(json.dumps({"escalations": None}), encoding="utf-8") + + skipped: set[str] = set() + found, _withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + + assert skipped == {str(fpath)} + assert found == [] + + +def test_gather_escalations_leaves_the_sink_optional(tmp_path): + """~20 rows call this reader with no sink, and the ordinary walk must not need + one. The default `None` is what keeps the signal additive rather than a second + contract every caller has to satisfy.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["raised"]]) + _unreadable_artifact(run_dir, task, 0) + + assert resolve._gather_escalations(run_dir, state, key) == ([], 0) + + +_POSIX_MODE_BITS = pytest.mark.skipif( + sys.platform == "win32", reason="Windows does not deny directory access by mode bits" +) +_NOT_ROOT = pytest.mark.skipif( + os.geteuid() == 0 if hasattr(os, "geteuid") else False, reason="root bypasses mode bits" +) +_FIFO = pytest.mark.skipif(not hasattr(os, "mkfifo"), reason="POSIX FIFOs") + + +@_POSIX_MODE_BITS +@_NOT_ROOT +def test_gather_escalations_records_an_artifact_it_cannot_stat(tmp_path): + """An artifact that EXISTS and cannot be reached is unreadable, not absent — and + which of those the old `Path.is_file()` probe reported depended on the interpreter, + so that one line carried two different defects at once. Through 3.13 it re-raises + EACCES (not in `pathlib._IGNORED_ERRNOS`), which escaped `build_context` and + `cmd_resolve` to `main`'s backstop as `error: [Errno 13] ...`, exit 1 — the exact + thing this reader's contract forbids. On 3.14 `is_file()` became `os.path.isfile`, + which swallows the error and answers False: the sink stays EMPTY, so the caller + reads a clean run, stamps `escalations_resolved_upto = len(task.sessions)`, and the + CRITICAL entries under that directory are withheld as already answered FOREVER. + + A real mode-000 parent, never a patched `Path.stat`: on 3.14 `is_file()` reaches + `os.stat`, so a mock on the pathlib method is never consulted and the row would + pass with the fix ablated — a false green on the one leg the second defect lives on. + + Both names in `TASK_CYCLE_ARTIFACTS` are recorded, which is the honest answer: with + the directory unreachable the reader cannot tell which of them was even there. + + The readable sibling is the positive control — the degrade still costs exactly the + directory it could not read. + + Ablation: restore `if not fpath.is_file(): continue` outside the `try` and this + reddens — on 3.13 with the PermissionError escaping, on 3.14 on an empty sink.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["older"], ["newer"]]) + task_dir = run_dir / "tasks" / task.sessions[1].task_id + task_dir.chmod(0o000) + try: + skipped: set[str] = set() + found, withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + + assert skipped == {str(task_dir / name) for name in TASK_CYCLE_ARTIFACTS} + assert [e["detail"] for e in found] == ["older"] # the sibling still read + assert withheld == 0 + + _path, _withheld, unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + assert unreadable == len(TASK_CYCLE_ARTIFACTS) # and it reaches the caller + finally: + task_dir.chmod(0o755) # so the sandbox tears down cleanly + + +def test_gather_escalations_treats_a_directory_at_the_artifact_path_as_absent(tmp_path): + """`stat` succeeds on a directory where `is_file()` answered False, so the mode + check is what keeps the switch behavior-preserving. Without it the directory + reaches `read_text`, raises `IsADirectoryError` — an `OSError` — and lands in the + sink, which withholds coverage over a path that holds no artifact at all. + + Ablation: drop the `S_ISREG` check and this reddens with the directory in the + sink.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["raised"]]) + (run_dir / "tasks" / task.sessions[0].task_id / "result.json").mkdir() + + skipped: set[str] = set() + found, _withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + + assert skipped == set() + assert [e["detail"] for e in found] == ["raised"] # the real artifact still read + + +@_FIFO +def test_gather_escalations_does_not_open_a_fifo_at_the_artifact_path(tmp_path): + """The dangerous half of the same mode check. `stat` succeeds on a FIFO too, so + without `S_ISREG` the walk would `read_text` it — and with no writer that blocks + FOREVER, wedging the interactive resolve command rather than failing it. The + classification never opens the path, which is why this row asserts through the + reader's answer and never reads the FIFO itself. + + Bounded with `SIGALRM`, following `test_diagnostics.py`'s twin: a hang is the + failure under test, so it needs a deadline or an ablation wedges the suite instead + of reddening it. + + Ablation: drop the `S_ISREG` check and the alarm fires.""" + import signal + + run_dir, state, task, key = _watermarked_trail(tmp_path, [["raised"]]) + os.mkfifo(run_dir / "tasks" / task.sessions[0].task_id / "result.json") + + def _blew_up(signum, frame): + raise AssertionError("the walk opened the FIFO instead of classifying it") + + previous = signal.signal(signal.SIGALRM, _blew_up) + signal.alarm(20) + try: + skipped: set[str] = set() + found, _withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + finally: + signal.alarm(0) + signal.signal(signal.SIGALRM, previous) + + assert skipped == set() + assert [e["detail"] for e in found] == ["raised"] + + +@pytest.mark.skipif(sys.platform == "win32", reason="POSIX symlinks") +def test_gather_escalations_records_a_symlink_loop_at_the_artifact_path(tmp_path): + """The one DELIBERATE behavior change in the switch to `stat`, and the row that + grades it. A symlink cycle where an artifact belongs is a degrade — something is + there and its contents cannot be reached — but `is_file()` called it ABSENT on + every supported interpreter: through 3.13 because ELOOP(40) is IN + `pathlib._IGNORED_ERRNOS`, and on 3.14 because `os.path.isfile` swallows it too. + Absent costs nothing, so the caller would stamp coverage over a session whose + escalations were never read. `stat` raises, and the fault joins the skip sink. + + MEASURED rather than argued, because the analogy nearby is a trap: `Path.resolve()` + on a loop raises `RuntimeError` — NOT an `OSError` — on 3.11 and raises nothing at + all on 3.13, so an arm catching `OSError` around IT would be inert. `Path.stat()` + is a syscall-level error and was probed uniform on 3.11.13, 3.13.14 and 3.14.6: + `OSError` errno 40 on all three, which is what makes one `except OSError` arm + enough. The premise below is asserted for the same reason. + + The readable sibling is the positive control, and unlike the EACCES row this one + reddens identically on every leg — the pre-fix answer was False everywhere. + + Ablation: restore `if not fpath.is_file(): continue` outside the `try` and this + reddens on an empty sink.""" + run_dir, state, task, key = _watermarked_trail(tmp_path, [["older"], ["newer"]]) + task_dir = run_dir / "tasks" / task.sessions[1].task_id + loop = task_dir / "result.json" + partner = task_dir / "result.json.cycle" + loop.symlink_to(partner) + partner.symlink_to(loop) + # The premise, MEASURED: the probe this fix replaced reported the cycle as absent, + # which is precisely the reading being changed. + assert not loop.is_file() + + skipped: set[str] = set() + found, withheld = resolve._gather_escalations(run_dir, state, key, skipped=skipped) + + assert skipped == {str(loop)} + assert [e["detail"] for e in found] == ["newer", "older"] # the siblings still read + assert withheld == 0 + + _path, _withheld, unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + assert unreadable == 1 # and it reaches the caller that decides coverage + + +def test_build_context_reports_the_unreadable_artifact_count(tmp_path): + """The third return member, from the same single walk. Zero whenever the run-dir + reads cleanly, which is why the coverage path is unchanged on an ordinary run. + + Ablation: return a constant 0 instead of `len(unreadable)` and this reddens on the + corrupt run while the clean one stays green.""" + run_dir, _state, task, key = _watermarked_trail(tmp_path, [["older"], ["newer"]]) + + _path, withheld, unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + assert (withheld, unreadable) == (0, 0) # the clean baseline + + _unreadable_artifact(run_dir, task, 1) + _path, _withheld, unreadable = resolve.build_context( + load_state(run_dir), run_dir, key, isolation="" + ) + assert unreadable == 1 # ----------------------------------------------------------- run_session @@ -2055,7 +3374,7 @@ def interactive_env(self, spec): def test_run_session_detects_resolution(tmp_path, monkeypatch): run_dir, state, _ = _escalated_run(tmp_path) - resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="") + _context(state, run_dir, "6-4-cli-list-command", isolation="") def fake_subprocess_run(argv, cwd, env): # simulate the agent writing the resolution marker @@ -2071,7 +3390,7 @@ def fake_subprocess_run(argv, cwd, env): def test_run_session_no_resolution(tmp_path, monkeypatch): run_dir, state, _ = _escalated_run(tmp_path) - resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="") + _context(state, run_dir, "6-4-cli-list-command", isolation="") monkeypatch.setattr(resolve.subprocess, "run", lambda *a, **k: None) assert ( resolve.run_session( @@ -2085,7 +3404,7 @@ def test_run_session_clears_stale_marker(tmp_path, monkeypatch): """A marker left by a previous resolve of this story must not be read as this session's output (the agent that says 'already resolved' writes none).""" run_dir, state, _ = _escalated_run(tmp_path) - resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="") + _context(state, run_dir, "6-4-cli-list-command", isolation="") stale = resolve.resolution_path(run_dir, "6-4-cli-list-command") stale.parent.mkdir(parents=True, exist_ok=True) stale.write_text('{"from": "last time"}', encoding="utf-8") @@ -2199,9 +3518,7 @@ def test_build_context_stories_carries_manifest_entry(tmp_path): run_dir, state, _ = _escalated_run(tmp_path, spec_file="/abs/spec.md", source="stories") state.spec_folder = "epic-1" - ctx = json.loads( - resolve.build_context(state, run_dir, key, isolation="").read_text(encoding="utf-8") - ) + ctx = json.loads(_context(state, run_dir, key, isolation="").read_text(encoding="utf-8")) st = ctx["stories"] assert st["spec_folder"] == "epic-1" assert st["story"]["title"] == "List command" @@ -2225,9 +3542,7 @@ def test_build_context_stories_sentinel_indicator(tmp_path): run_dir, state, _ = _escalated_run(tmp_path, spec_file=str(sentinel), source="stories") state.spec_folder = "epic-1" - ctx = json.loads( - resolve.build_context(state, run_dir, key, isolation="").read_text(encoding="utf-8") - ) + ctx = json.loads(_context(state, run_dir, key, isolation="").read_text(encoding="utf-8")) sent = ctx["stories"]["sentinel"] assert sent["kind"] == "unresolved" assert "intent too vague" in sent["blocking_condition"] @@ -2237,9 +3552,7 @@ def test_build_context_sprint_mode_has_no_stories_block(tmp_path): """Sprint mode leaves the context contract unchanged — no stories block.""" run_dir, state, _ = _escalated_run(tmp_path, spec_file="/abs/spec.md") # sprint source ctx = json.loads( - resolve.build_context(state, run_dir, "6-4-cli-list-command", isolation="").read_text( - encoding="utf-8" - ) + _context(state, run_dir, "6-4-cli-list-command", isolation="").read_text(encoding="utf-8") ) assert "stories" not in ctx @@ -2261,9 +3574,9 @@ def test_build_context_leaves_an_out_of_mount_spec_unchanged(tmp_path): run_dir, state, _ = _escalated_run(tmp_path, spec_file=str(spec), worktree_path=str(wt)) ctx = json.loads( - resolve.build_context( - state, run_dir, "6-4-cli-list-command", isolation="worktree" - ).read_text(encoding="utf-8") + _context(state, run_dir, "6-4-cli-list-command", isolation="worktree").read_text( + encoding="utf-8" + ) ) assert ctx["spec_file"] == spec.as_posix() @@ -2304,7 +3617,7 @@ def test_build_context_stories_block_names_the_same_tree_as_spec_file(tmp_path): state.spec_folder = "epic-1" ctx = json.loads( - resolve.build_context(state, run_dir, key, isolation="worktree").read_text(encoding="utf-8") + _context(state, run_dir, key, isolation="worktree").read_text(encoding="utf-8") ) assert ctx["spec_file"] == (wt / rel).as_posix() sent = ctx["stories"]["sentinel"] @@ -2353,7 +3666,7 @@ def test_build_context_stories_block_stays_on_the_mount_for_an_out_of_mount_spec state.spec_folder = "epic-1" ctx = json.loads( - resolve.build_context(state, run_dir, key, isolation="worktree").read_text(encoding="utf-8") + _context(state, run_dir, key, isolation="worktree").read_text(encoding="utf-8") ) assert ctx["spec_file"] == outside.as_posix() # unchanged: absolute passes through sent = ctx["stories"]["sentinel"] @@ -2375,9 +3688,9 @@ def test_build_context_reports_whether_the_spec_reaches_the_redrive(tmp_path): wt = tmp_path / ".bmad-loop" / "runs" / "20260613-111429-6a14" / "worktrees" / "1" run_dir, state, _ = _escalated_run(tmp_path, spec_file="specs/6-4.md", worktree_path=str(wt)) ctx = json.loads( - resolve.build_context( - state, run_dir, "6-4-cli-list-command", isolation="worktree" - ).read_text(encoding="utf-8") + _context(state, run_dir, "6-4-cli-list-command", isolation="worktree").read_text( + encoding="utf-8" + ) ) assert ctx["spec_reaches_the_redrive"] is False @@ -2385,9 +3698,9 @@ def test_build_context_reports_whether_the_spec_reaches_the_redrive(tmp_path): tmp_path, "20260613-111429-6a15", spec_file=str(tmp_path / "specs" / "6-4.md") ) plain = json.loads( - resolve.build_context( - plain_state, plain_dir, "6-4-cli-list-command", isolation="" - ).read_text(encoding="utf-8") + _context(plain_state, plain_dir, "6-4-cli-list-command", isolation="").read_text( + encoding="utf-8" + ) ) assert plain["spec_reaches_the_redrive"] is True @@ -2410,9 +3723,9 @@ def test_build_context_names_where_an_unreachable_correction_has_to_land(tmp_pat run_dir, state, _ = _escalated_run(tmp_path, spec_file="specs/6-4.md", worktree_path=str(wt)) state.target_branch = "feat/the-pinned-one" ctx = json.loads( - resolve.build_context( - state, run_dir, "6-4-cli-list-command", isolation="worktree" - ).read_text(encoding="utf-8") + _context(state, run_dir, "6-4-cli-list-command", isolation="worktree").read_text( + encoding="utf-8" + ) ) # the paired claim: the edit has no future, and THIS is the tree that does assert ctx["spec_reaches_the_redrive"] is False @@ -2424,9 +3737,9 @@ def test_build_context_names_where_an_unreachable_correction_has_to_land(tmp_pat ) plain_state.target_branch = "feat/the-pinned-one" # set, but no mount to make it apply plain = json.loads( - resolve.build_context( - plain_state, plain_dir, "6-4-cli-list-command", isolation="" - ).read_text(encoding="utf-8") + _context(plain_state, plain_dir, "6-4-cli-list-command", isolation="").read_text( + encoding="utf-8" + ) ) assert plain["redrive_base_ref"] == "HEAD" @@ -2473,7 +3786,7 @@ def test_rearm_warns_about_an_unreachable_spec_write_only_when_it_is_actionable( run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) unreachable = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert bool(unreachable) is warns @@ -2545,7 +3858,7 @@ def _commit(status, message): ) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) unreachable = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert bool(unreachable) is warns @@ -2604,7 +3917,7 @@ def _sentinel_run( sentinel = folder / f"{key}-unresolved.md" sentinel.write_text( - "---\nstatus: blocked\n---\n\n## Auto Run Result\n\n" "Status: blocked\nintent too vague\n", + "---\nstatus: blocked\n---\n\n## Auto Run Result\n\nStatus: blocked\nintent too vague\n", encoding="utf-8", ) mount = tmp_path / "wt" @@ -2668,7 +3981,7 @@ def test_rearm_holds_a_sentinel_until_the_upstream_correction_reaches_the_redriv ) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=isolated) + runs.rearm_escalation(run_dir, isolated_redrive=isolated, resolution_recorded=True) assert not sentinel.exists() # the sentinel really was cleared on every row records = _upstream_records(run_dir) @@ -2757,7 +4070,7 @@ def _commit(intent, message): ) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) records = _upstream_records(run_dir) assert bool(records) is warns @@ -2792,7 +4105,7 @@ def test_rearm_exempts_a_stories_folder_configured_outside_the_project( ) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) assert bool(_upstream_records(run_dir)) is not external @@ -2834,7 +4147,9 @@ def test_rearm_of_a_sentinel_survives_a_project_that_is_not_a_repository(tmp_pat ) monkeypatch.chdir(tmp_path) - assert runs.rearm_escalation(run_dir, isolated_redrive=True) == key # no GitError + assert ( + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) == key + ) # no GitError assert not sentinel.exists() # the destructive half still completed (rec,) = _upstream_records(run_dir) @@ -2886,7 +4201,7 @@ def test_rearm_records_the_in_place_remedy_when_isolation_was_turned_off(tmp_pat monkeypatch.chdir(tmp_path) # the flip: policy now says `none`, while the recorded mount still says otherwise - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) (rec,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert rec["redrive"] == "in-place" @@ -2943,7 +4258,7 @@ def test_rearm_in_place_proof_reads_the_working_tree_not_the_commit(tmp_path, mo root, spec_file=rel, worktree_path=str(mount), target_branch="main" ) monkeypatch.chdir(root) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) fired = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert bool(fired) is warns, f"corrected={corrected}" @@ -3004,7 +4319,7 @@ def test_rearm_base_ref_degrades_to_head_for_a_run_that_pinned_no_target(tmp_pat run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] == [] @@ -3057,7 +4372,7 @@ def test_rearm_does_not_refuse_a_flip_the_redrive_never_reads( monkeypatch.chdir(tmp_path) runs.rearm_escalation( - run_dir, isolated_redrive=True + run_dir, isolated_redrive=True, resolution_recorded=True ) # must not raise: this flip cannot reach the re-drive kinds = _kinds(run_dir) @@ -3090,7 +4405,7 @@ def test_rearm_suppresses_the_unreachable_warning_only_on_proof(tmp_path, monkey run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel, worktree_path=str(tmp_path / "wt")) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) kinds = _kinds(run_dir) (unreachable,) = [e for e in kinds if e["kind"] == "rearm-spec-write-unreachable"] @@ -3132,7 +4447,7 @@ def test_rearm_does_not_warn_when_the_spec_dir_is_shared_with_the_redrive(tmp_pa ) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) assert [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] == [] # and the flip really landed on the shared file the re-drive will read @@ -3176,7 +4491,7 @@ def test_rearm_still_warns_for_a_spec_spelled_out_of_but_resolving_into_the_work run_dir, _, _ = _escalated_run(tmp_path, spec_file=str(spelled), worktree_path=str(wt)) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) (unreachable,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert unreachable["status"] == "ready-for-dev" @@ -3215,7 +4530,7 @@ def _refuse(self, *a, **kw): ) monkeypatch.chdir(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=True) + runs.rearm_escalation(run_dir, isolated_redrive=True, resolution_recorded=True) (unreachable,) = [e for e in _kinds(run_dir) if e["kind"] == "rearm-spec-write-unreachable"] assert unreachable["status"] == "ready-for-dev" @@ -3253,7 +4568,7 @@ def test_rearm_writes_the_project_rooted_spec_when_no_worktree_was_recorded(tmp_ run_dir, _, _ = _escalated_run(tmp_path, spec_file=rel) # worktree_path="" -> the fallback monkeypatch.chdir(tmp_path / "elsewhere") - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) fm = verify.read_frontmatter(spec) assert fm["status"] == "ready-for-dev" # the project-rooted copy was flipped @@ -3324,6 +4639,126 @@ def test_rearm_event_notice_splits_the_flip_skip_on_the_refusal(): assert step == "" +def test_rearm_event_notice_splits_the_abort_three_ways_on_the_rollback(): + """One kind, THREE renderings, and the split is by what the surface may CLAIM about + the file — not by how the re-arm failed. + + Nothing else grades this. The CLI's abort-echo test asserts "the re-arm ABORTED", + "nothing was persisted", "still escalated" and the spec path, and every one of those + is true of the `failed` message too — so replacing the discriminator with `if False:` + was SILENT across the whole suite while an operator holding a part-written spec was + told it had been "left exactly as the re-arm found it". A row that reads the table + directly is the only place the three can be compared. + + The unrecognized-value leg is the load-bearing one. `unknown` is a real producer + answer (the sentinel-clear leg, and a spec the undo could not read), an absent field + is what a record from an older or future producer looks like, and neither may inherit + the reassuring branch by falling through to it. So the default is the branch that + claims nothing, and the assertions below say that in the strongest available form: + the "left exactly as the re-arm found it" sentence appears on `restored`/`unchanged` + and NOWHERE else. + + next_step is graded beside the message for `rearm-spec-flip-skipped`'s reason above — + it is the half that costs an operator time — and for one this kind adds: the TUI + drops next_step entirely, so `failed`'s restore-from-git remedy has to survive in the + MESSAGE as well. That is asserted on the message, not just on the step. + + Ablations, each run: replace the `rollback == "failed"` test with `if False:` and the + `failed` assertions redden; replace the `rollback in ("restored", "unchanged")` test + with `if True:` and the `unknown`/absent assertions redden; drop "restore it from git" + from the `failed` MESSAGE (keeping next_step) and the TUI-reachability assertion + reddens alone. + """ + entry = { + "kind": "rearm-aborted", + "spec_file": "/p/specs/s1.md", + "error": "OSError: [Errno 28] No space left on device", + } + left_as_found = "left exactly as the re-arm found it" + + _, failed_msg, failed_step = runs.rearm_event_notice({**entry, "rollback": "failed"}) + _, restored_msg, restored_step = runs.rearm_event_notice({**entry, "rollback": "restored"}) + _, unchanged_msg, _ = runs.rearm_event_notice({**entry, "rollback": "unchanged"}) + _, unknown_msg, unknown_step = runs.rearm_event_notice({**entry, "rollback": "unknown"}) + _, absent_msg, absent_step = runs.rearm_event_notice(entry) + _, future_msg, _ = runs.rearm_event_notice({**entry, "rollback": "something-new"}) + + # every rendering states the two facts that are true whatever happened + for msg in (failed_msg, restored_msg, unchanged_msg, unknown_msg, absent_msg, future_msg): + assert "the re-arm ABORTED" in msg + assert "nothing was persisted" in msg + assert "still escalated" in msg + + # ...and ONLY the two outcomes that proved it say the file is intact + assert left_as_found in restored_msg and left_as_found in unchanged_msg + assert left_as_found not in failed_msg + assert left_as_found not in unknown_msg + assert left_as_found not in absent_msg + assert left_as_found not in future_msg # an unrecognized value defaults to NOT reassuring + + # `failed` is the only one that can leave a part-written spec, and its remedy has to + # reach a TUI operator, which never sees next_step + assert "may be left part-written" in failed_msg + assert "restore it from git" in failed_msg + # ...and it names a SECOND source, because the bytes the undo failed to write are gone + # with the process and an untracked or out-of-checkout spec has no committed copy + assert "or from your own copy" in failed_msg + assert failed_step == "Restore the spec from git or your own copy, then re-run resolve" + # ...and it does NOT enumerate which writes landed: a fault inside + # `strip_auto_run_result` reaches the guard with the flip published and the section + # still present, so an enumeration would describe a state this record cannot know + assert "## Auto Run Result" not in failed_msg + + # the three next_steps are distinct remedies, not one sentence reused + assert len({failed_step, restored_step, unknown_step}) == 3 + assert absent_step == unknown_step # an absent field IS the unknown outcome + + +def test_rearm_event_notice_renders_the_commits_probe_failure(): + """The row that stops a FAILED commits probe reading as a clean one (DW-81). + + `stale-restore-commits` is written only when the probe answered, so its absence + used to carry two opposite meanings — "nothing from the abandoned attempt" and + "nobody could tell" — and neither operator surface could separate them. This row + is the separation, so it is graded on all three returned fields: + + * the truncated baseline, because that is the ref the operator has to diff from + and the record is read out of process from the journal line alone; + * the typed error, because a bad baseline and a non-repo code tree are different + things to go fix; + * the range, in the MESSAGE as well as the next_step — the TUI drops `next_step` + and resumes in the same gesture, so a message that only said "something went + wrong" would leave that surface's operator with no action at all. + + Ablation: return None for this kind and every assertion here reddens; drop the + `git log` range from the message while keeping it in the next_step and only the + message assertion does — which is the half the TUI would have lost. + """ + baseline = "abc123def456" + "0" * 28 + rec = { + "kind": "rearm-commits-probe-failed", + "story_key": "1-1-a", + "old_baseline": baseline, + "error": f"GitError: git rev-list {baseline}..HEAD failed in /code:\n" + + "fatal " + + "x" * 5000, + } + severity, message, next_step = runs.rearm_event_notice(rec) + assert severity == "warning" + assert "abc123def456.." in message # truncated to 12, as the sibling row does + assert "0" * 28 not in message # ...and NOT the whole sha + assert "GitError" in message and "rev-list" in message # the typed cause + assert "\n" not in message and len(message) < 4500 # terminal-safe and bounded + assert "proves nothing" in message # the silence is not evidence of "clean" + assert "fix the Git failure" in message # do not blindly repeat the failed probe + assert "git log abc123def456..HEAD" in message # actionable on the TUI alone + assert next_step == ( + "Fix the Git failure, then check `git log abc123def456..HEAD` before resuming" + ) + # the imperative lives ONLY in next_step: the TUI drops it and resumes here + assert "before resuming" not in message + + def test_rearm_holds_the_resume_only_on_the_record_that_proves_a_wedge(): """The hold is PROOF, not urgency — and it is asked of every kind the table knows. @@ -3349,6 +4784,9 @@ def test_rearm_holds_the_resume_only_on_the_record_that_proves_a_wedge(): "stale-restore-commits", "stale-restore-unparseable", "stale-restore-excluded", + # the probe that could NOT answer proves strictly less than the answer, so if + # `stale-restore-commits` does not hold, neither can this + "rearm-commits-probe-failed", "rearm-baseline-advance-failed", "rearm-baseline-restamp-skipped", "rearm-baseline-restamped", diff --git a/tests/test_runs.py b/tests/test_runs.py index 6eaf397d..dffba09d 100644 --- a/tests/test_runs.py +++ b/tests/test_runs.py @@ -1,6 +1,7 @@ """Run-directory helper tests.""" import contextlib +import errno import json import os import re @@ -2791,7 +2792,12 @@ def test_rearm_restore_mode_sets_in_review_strips_arr_and_latches(tmp_path): from bmad_loop.model import Phase run_dir, spec = _escalated_run(tmp_path, _SPEC_WITH_ARR) - runs.rearm_escalation(run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False) + runs.rearm_escalation( + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, + ) task = load_state(run_dir).tasks["1-1-a"] assert task.phase == Phase.PENDING and task.attempt == 0 @@ -2809,7 +2815,9 @@ def test_rearm_plain_mode_sets_ready_for_dev_and_clears_stale_latch(tmp_path): # a stale latch from a prior restore attempt the human then chose to redo fresh run_dir, spec = _escalated_run(tmp_path, _SPEC_WITH_ARR, restore_patch_stale="old.patch") - runs.rearm_escalation(run_dir, isolated_redrive=False) # no restore_patch => from-scratch + runs.rearm_escalation( + run_dir, isolated_redrive=False, resolution_recorded=True + ) # no restore_patch => from-scratch task = load_state(run_dir).tasks["1-1-a"] assert task.phase == Phase.PENDING @@ -2840,7 +2848,10 @@ def test_rearm_aborts_when_the_spec_status_cannot_be_reopened(tmp_path): with pytest.raises(runs.RearmError, match="re-open story spec"): runs.rearm_escalation( - run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, ) assert spec.read_text(encoding="utf-8") == spec_text # byte-identical @@ -2860,7 +2871,7 @@ def test_rearm_resets_followup_reviews_spent(tmp_path): state.tasks["1-1-a"].review_cycle = 2 save_state(run_dir, state) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["1-1-a"] assert task.followup_reviews_spent == 0 @@ -2902,10 +2913,23 @@ def _kinds(run_dir, prefix="stale-restore-"): def test_rearm_excludes_stale_restore_residue_from_baseline_snapshot(tmp_path): """The abandoned attempt's applied new files must NOT be blessed as pre-existing, or finalize_commit's `add -A` sweeps them into the corrected - story's commit. The resolve session's own untracked file still is.""" + story's commit. The resolve session's own untracked file still is. + + Also the commits probe's ORDINARY answer: nothing was committed above the old + baseline here, so `verify.commits_above` returns `[]` and BOTH commit records + stay away. That silence is the one an operator is entitled to read as "clean", + which is exactly why the failed probe now writes `rearm-commits-probe-failed` + instead of reproducing it (DW-81). + + Ablation: relax the producer's `if shas:` gate to `if shas is not None:` and the + `stale-restore-commits` assertion reddens; journal the probe failure outside its + `except` arm and the `rearm-commits-probe-failed` one does. + """ run_dir, _spec, patch = _stale_restore_tree(tmp_path) - runs.rearm_escalation(run_dir, isolated_redrive=False) # from-scratch re-arm replaces the latch + runs.rearm_escalation( + run_dir, isolated_redrive=False, resolution_recorded=True + ) # from-scratch re-arm replaces the latch task = load_state(run_dir).tasks["1-1-a"] assert "human.txt" in task.baseline_untracked @@ -2915,6 +2939,9 @@ def test_rearm_excludes_stale_restore_residue_from_baseline_snapshot(tmp_path): assert len(excluded) == 1 assert excluded[0]["files"] == ["newfile.txt"] assert excluded[0]["patch"] == str(patch) + # the probe ran and answered "none" — neither commit record may appear + assert not _kinds(run_dir, "stale-restore-commits") + assert not _kinds(run_dir, "rearm-commits-probe-failed") def test_rearm_re_latching_the_same_patch_still_excludes_its_residue(tmp_path): @@ -2922,7 +2949,12 @@ def test_rearm_re_latching_the_same_patch_still_excludes_its_residue(tmp_path): still residue (and `git apply` would otherwise fail with 'already exists').""" run_dir, _spec, _patch = _stale_restore_tree(tmp_path) - runs.rearm_escalation(run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False) + runs.rearm_escalation( + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, + ) task = load_state(run_dir).tasks["1-1-a"] assert task.restore_patch == "artifacts/attempt.patch" @@ -2940,7 +2972,9 @@ def test_rearm_missing_stale_patch_degrades_loudly_without_raising(tmp_path): git(tmp_path, "add", "committed.txt") git(tmp_path, "commit", "-q", "-m", "attempt commit") - runs.rearm_escalation(run_dir, isolated_redrive=False) # must not raise RearmError + runs.rearm_escalation( + run_dir, isolated_redrive=False, resolution_recorded=True + ) # must not raise RearmError task = load_state(run_dir).tasks["1-1-a"] assert {"human.txt", "newfile.txt"} <= set(task.baseline_untracked) # full snapshot @@ -2956,7 +2990,12 @@ def test_rearm_without_a_stale_latch_journals_no_stale_restore_events(tmp_path): run_dir, _spec = _escalated_run(tmp_path, _SPEC_WITH_ARR, git_project=True) (tmp_path / "human.txt").write_text("from the resolve session\n") - runs.rearm_escalation(run_dir, restore_patch="artifacts/attempt.patch", isolated_redrive=False) + runs.rearm_escalation( + run_dir, + restore_patch="artifacts/attempt.patch", + isolated_redrive=False, + resolution_recorded=True, + ) assert "human.txt" in load_state(run_dir).tasks["1-1-a"].baseline_untracked assert _kinds(run_dir) == [] @@ -2972,7 +3011,7 @@ def test_rearm_warns_about_commits_below_the_refreshed_baseline(tmp_path): git(tmp_path, "commit", "-q", "-m", "attempt commit") old_baseline = load_state(run_dir).tasks["1-1-a"].baseline_commit - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["1-1-a"] assert task.baseline_commit != old_baseline # baseline advanced past the commit @@ -2984,10 +3023,19 @@ def test_rearm_warns_about_commits_below_the_refreshed_baseline(tmp_path): def test_rearm_survives_a_git_fault_reading_commits_above_the_old_baseline(tmp_path): """A bad old baseline is warn-only, and the persisted reset proves re-arm - reached its save rather than returning early. + reached its save rather than returning early — and it now leaves a RECORD. + + The probe failing used to be byte-identical to the probe finding nothing: both + wrote no journal entry, so `assert not _kinds(run_dir, "stale-restore-commits")` + below is true for two opposite reasons and cannot tell them apart. The + `rearm-commits-probe-failed` assertions are what separate them (DW-81) — without + them this test passes on a re-arm that silently swallowed the fault. Ablation: catch a type outside ``verify.GitError`` and the real rev-list - failure escapes before any of these completion assertions can run. + failure escapes before any of these completion assertions can run. Delete the + new ``journal.append("rearm-commits-probe-failed", ...)`` and the length + assertion below reddens while every pre-existing assertion here stays green — + which is the gap it was added to close. """ from bmad_loop.model import Phase @@ -2998,7 +3046,7 @@ def test_rearm_survives_a_git_fault_reading_commits_above_the_old_baseline(tmp_p task.baseline_commit = "0" * 39 + "1" # sha-shaped, but names no object save_state(run_dir, state) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["1-1-a"] assert task.phase == Phase.PENDING @@ -3006,17 +3054,28 @@ def test_rearm_survives_a_git_fault_reading_commits_above_the_old_baseline(tmp_p assert task.generation == initial_generation + 1 assert task.restore_patch is None assert task.baseline_commit == git(tmp_path, "rev-parse", "HEAD") - assert not _kinds(run_dir, "stale-restore-commits") + assert not _kinds(run_dir, "stale-restore-commits") # the probe never answered... + probe = _kinds(run_dir, "rearm-commits-probe-failed") # ...and now says so + assert len(probe) == 1 + assert probe[0]["old_baseline"] == "0" * 39 + "1" # the baseline it could not read + assert probe[0]["story_key"] == "1-1-a" + # the typed error, spelled the way the sibling `rearm-baseline-advance-failed` + # spells it — `GitError: ...`, not a bare repr + assert probe[0]["error"].startswith("GitError: ") + assert "rev-list" in probe[0]["error"] excluded = _kinds(run_dir, "stale-restore-excluded") assert len(excluded) == 1 assert excluded[0]["files"] == ["newfile.txt"] def test_rearm_survives_a_non_repo_code_tree_when_reading_commits(tmp_path): - """A non-repository code tree reaches the same typed, silent degrade. + """A non-repository code tree reaches the same typed, warn-only degrade — and + the same record, because the operator's exposure is identical either way. Ablation: catch a type outside ``verify.GitError`` and the pinned probe fault - escapes, so the persisted generation and latch reset never appear. + escapes, so the persisted generation and latch reset never appear. Delete the + new ``journal.append("rearm-commits-probe-failed", ...)`` and only the record + assertions redden. """ from bmad_loop.model import Phase @@ -3029,7 +3088,7 @@ def test_rearm_survives_a_non_repo_code_tree_when_reading_commits(tmp_path): with pytest.raises(verify.GitError): verify.commits_above(tmp_path, task.baseline_commit) - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) task = load_state(run_dir).tasks["1-1-a"] assert task.phase == Phase.PENDING @@ -3038,23 +3097,780 @@ def test_rearm_survives_a_non_repo_code_tree_when_reading_commits(tmp_path): assert task.restore_patch is None assert task.baseline_commit == "0" * 39 + "1" assert not _kinds(run_dir, "stale-restore-commits") + probe = _kinds(run_dir, "rearm-commits-probe-failed") + assert len(probe) == 1 + assert probe[0]["old_baseline"] == "0" * 39 + "1" + assert probe[0]["error"].startswith("GitError: ") + assert len(_kinds(run_dir, "stale-restore-unparseable")) == 1 + + +def test_rearm_skips_the_commits_probe_entirely_without_a_recorded_baseline(tmp_path): + """No recorded baseline means no range to ask about, so the probe never runs — + and a probe that never ran must not journal that it FAILED. + + The `if old_baseline:` guard is what separates "there was nothing to measure + against" from "the measurement broke", and `rearm-commits-probe-failed` claims + the second. Telling an operator to go diff a range that was never established + would be the mirror of the silence DW-81 closed: a warning with no referent, + trained straight into the scroll-past habit the `restore` split exists to prevent. + + The sibling `stale-restore-unparseable` is asserted PRESENT on purpose: without + it this test is green for the uninteresting reason that + `_stale_restore_residue` returned early on a missing latch and journalled + nothing at all. It proves the function ran and only the commits block was + skipped. + + Ablation: delete the `if old_baseline:` guard and `commits_above` is handed a + `None` baseline, git fails on the `None..HEAD` range, and the record this test + denies appears. + """ + run_dir, _spec = _escalated_run(tmp_path, _SPEC_WITH_ARR, restore_patch_stale="old.patch") + assert load_state(run_dir).tasks["1-1-a"].baseline_commit is None # pin the premise + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert not _kinds(run_dir, "rearm-commits-probe-failed") + assert not _kinds(run_dir, "stale-restore-commits") + # ...while the residue pass itself did run: the latched patch is missing assert len(_kinds(run_dir, "stale-restore-unparseable")) == 1 def test_rearm_does_not_swallow_a_non_git_fault_from_the_commits_probe(monkeypatch, tmp_path): - """Only Git faults are warn-only; programming faults must escape. + """Only Git faults are warn-only; programming faults must escape — and the re-arm + they escape from leaves NOTHING behind. + + The propagation half graded the narrowing and stopped there, which made it silent on + the state the escape left: this probe runs after the status flip and the + `## Auto Run Result` strip have both published and before `save_state`, so the fault + used to exit with the spec re-armed on disk against a task the run still calls + ESCALATED — the one edit nothing else records (DW-79/DW-83). The window is one + transaction now, so the same fault also has to come back byte-identical. + + `generation` and `restore_patch` are read from the RELOADED task, not the object the + call mutated: `rearm_escalation` bumps both in memory long before the guard, and + `save_state` is the only thing that would have made them true. Asserting on the + in-memory task would pass with the whole transaction deleted. + + Ablations: widen the catch back to ``Exception`` and this fails with + ``DID NOT RAISE``, grading the narrowing; delete the `except BaseException` arm from + `rearm_escalation` and the spec-bytes assertion reddens while the raise still passes. + """ + from bmad_loop.model import Phase - Ablation: widen the catch back to ``Exception`` and this fails with - ``DID NOT RAISE``, directly grading the narrowing rather than its old behavior. + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + was = load_state(run_dir).tasks["1-1-a"] + + def boom(repo, baseline): + raise MemoryError("not a git answer") + + monkeypatch.setattr(runs.verify, "commits_above", boom) + with pytest.raises(MemoryError, match="not a git answer"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before # flip AND strip both undone + task = load_state(run_dir).tasks["1-1-a"] + assert task.phase == Phase.ESCALATED # nothing was persisted, so it is still armed + assert task.generation == was.generation + assert task.restore_patch == was.restore_patch + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "restored" # a published write really was put back + assert "MemoryError" in aborted["error"] + assert aborted["spec_file"] == str(spec) + # ...and the degrade record is NOT one of the things it leaves behind: this fault + # is not a git answer, so the warn-only arm never runs and the abort is the whole + # story. Graded by the same narrowing as the raise above — widen the catch to + # `Exception` and the fault is swallowed into a record instead of propagating. + assert not _kinds(run_dir, "rearm-commits-probe-failed") + + +def test_rearm_rolls_back_when_save_state_itself_fails(monkeypatch, tmp_path): + """`save_state` IS the commit point, so a fault raised BY it is the sharpest case + the transaction exists for: every spec write has landed and the one thing that would + make them true has not. + + It was also outside every undo the function used to carry — those sat in two `except` + arms further up — so an ENOSPC here left the spec flipped and stripped while + `state.json` still described an escalated story, with nothing on the record at all. + + The state file is compared BYTE-for-byte rather than by reloading and checking the + phase: a phase check passes for every reason a write could be absent, while the bytes + also grade the `generation` bump and the cleared `defer_reason` riding in the same + object. + + Ablation: delete the `except BaseException` arm and the spec bytes and the abort + record both redden; the OSError still propagates, which is why it alone is no oracle. """ - run_dir, _spec, _patch = _stale_restore_tree(tmp_path) + from bmad_loop.journal import STATE_FILE + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + state_before = (run_dir / STATE_FILE).read_bytes() + + def boom(run_dir_, state_): + raise OSError(28, "No space left on device") + + monkeypatch.setattr(runs, "save_state", boom) + with pytest.raises(OSError, match="No space left on device"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before + assert (run_dir / STATE_FILE).read_bytes() == state_before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "restored" + assert "OSError" in aborted["error"] + + +def test_rearm_rolls_back_when_the_window_is_interrupted(monkeypatch, tmp_path): + """The guard catches `BaseException`, and the breadth is load-bearing rather than + defensive — nothing else in the suite grades it. + + `KeyboardInterrupt` and `SystemExit` derive from `BaseException` alone, so narrowing + the arm to `except Exception` passes every other test while reopening the exact + DW-79/DW-83 state on the most ordinary operator gesture there is. The window is + mostly blocking I/O: three git subprocesses (`rev_parse_head`, `untracked_files`, + `commits_above`) and then `save_state`, all AFTER the status flip has published and + BEFORE anything persists it. A Ctrl-C in there under a narrowed arm exits with the + spec re-armed on disk against a task still recorded as ESCALATED. + + Raised from `save_state` because that is the last statement inside the guard, so the + interrupt lands at the widest point of the exposure — every spec write behind it and + the commit point not yet reached. + + Ablation (run): narrow the arm to `except Exception` and this reddens on the + spec-bytes assertion, which is simply the first of the four to run — remove the three + tree/state assertions and the abort record reddens behind them with + `ValueError: not enough values to unpack`, because with the arm narrowed no + `rearm-aborted` entry is written at all. The `KeyboardInterrupt` still propagates + either way, which is exactly why the raise alone is no oracle. + """ + from bmad_loop.journal import STATE_FILE + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + state_before = (run_dir / STATE_FILE).read_bytes() + + def interrupted(run_dir_, state_): + raise KeyboardInterrupt + + monkeypatch.setattr(runs, "save_state", interrupted) + with pytest.raises(KeyboardInterrupt): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before + assert (run_dir / STATE_FILE).read_bytes() == state_before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "restored" + assert "KeyboardInterrupt" in aborted["error"] + + +def test_rearm_abort_on_the_sentinel_leg_claims_nothing_about_the_file(monkeypatch, tmp_path): + """The sentinel clear is the one in-window tree change the transaction does NOT undo, + so the abort record must not describe the tree as untouched. + + That leg deletes the sentinel rather than writing spec bytes, and re-creating it would + fight a gesture that is already safe to repeat — `_clear_sentinel` preserves a copy + under `{run_dir}/sentinels/` and a retried resolve re-clears it idempotently. What was + wrong was never the deletion; it was the CLAIM. `spec_before` is `None` on this leg, + and folding that into `unchanged` made the notice name a file this re-arm had just + DELETED as proof nothing moved. + + So the outcome is `unknown`, and the rendering is graded here rather than only the + field: the field is what the producer wrote, the sentence is what the operator reads. + + Ablation (run): make `_restore_rearmed_spec` answer `"unchanged"` for + `original is None` and this reddens on the recorded `rollback` first; drop that one + assertion and the RENDERED message reddens behind it, on a notice that now reads + "(…1-1-a-unresolved.md) was left exactly as the re-arm found it" about a file this + re-arm deleted. Both halves are asserted because the field and the sentence are + different claims. The deletion assertions keep passing throughout, which is why they + alone do not grade this. + """ + from bmad_loop.journal import STATE_FILE + from bmad_loop.model import Phase + + sentinel = tmp_path / "1-1-a-unresolved.md" + sentinel.write_text("---\nstatus: blocked\n---\n\nplanning halted\n", encoding="utf-8") + run = escalated_run( + tmp_path, + "r1", + story_key="1-1-a", + source="stories", + sentinel_kind="unresolved", + spec_file=str(sentinel), + git_project=True, + ) + run_dir = run.run_dir + state_before = (run_dir / STATE_FILE).read_bytes() + + def boom(run_dir_, state_): + raise OSError(28, "No space left on device") + + monkeypatch.setattr(runs, "save_state", boom) + with pytest.raises(OSError, match="No space left on device"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + # the deletion STANDS — it is deliberately outside the transaction — and the + # preserved copy is why that is safe + assert not sentinel.exists() + assert (run_dir / "sentinels" / sentinel.name).is_file() + # ...while everything the transaction DOES cover was rolled back + assert (run_dir / STATE_FILE).read_bytes() == state_before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "unknown" + _severity, message, _next_step = runs.rearm_event_notice(aborted) + assert "the re-arm ABORTED" in message + assert "still escalated" in message + # the whole point: no claim about a file that is no longer there + assert "left exactly as the re-arm found it" not in message + + +def test_rearm_rolls_back_when_a_mid_window_journal_append_fails(monkeypatch, tmp_path): + """A `journal.append` inside the residue pass is an ordinary file write and can fail + like one — and it is the fault source furthest from anything that looks like a spec + write, which is exactly why no per-arm undo ever covered it. + + Only the residue kind is made to fail, so the abort record itself can still land: the + point being graded is that a fault from a helper that writes no spec still rolls the + spec back and still says so. + + Ablation: delete the `except BaseException` arm and the spec-bytes assertion reddens. + """ + from bmad_loop.journal import Journal + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + real_append = Journal.append + + def flaky(self, kind, **fields): + if kind == "stale-restore-excluded": + raise OSError(5, "Input/output error") + return real_append(self, kind, **fields) + + monkeypatch.setattr(Journal, "append", flaky) + with pytest.raises(OSError, match="Input/output error"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "restored" + + +def test_rearm_records_unchanged_when_the_sequenced_refusal_fires(tmp_path): + """`rollback: "unchanged"` is a DIFFERENT fact from `"restored"`, and the surfaces + render it differently, so the flip's read-back refusal has to produce it. + + That refusal is sequenced ahead of every write — `set_frontmatter_status` decides it + cannot move a spec with no top-level `status:` before it writes anything, and the + `## Auto Run Result` strip is deliberately ordered after the check — so the guard + finds the spec exactly as `spec_before` captured it and rewrites nothing. Recording + that as `restored` would tell an operator a write had landed and been undone on the + one path where nothing was ever written. + + Ablation: make `_restore_rearmed_spec`'s "bytes equal to `original`" arm return True + — the arm this path actually takes — and this reddens on the `rollback` value alone, + while every other assertion still passes. Its `original is None` arm does NOT grade + this row: the spec here is readable, so `spec_before` is set and that arm never runs. + """ + from bmad_loop.model import Phase + + run_dir, spec = _escalated_run( + tmp_path, "---\ntitle: t\n---\n\n## Intent\n\nbody\n\n## Auto Run Result\n\nx\n" + ) + before = spec.read_bytes() + + with pytest.raises(runs.RearmError, match="no frontmatter `status:`"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "unchanged" + assert "RearmError" in aborted["error"] + + +def test_rearm_reports_a_rollback_that_itself_failed_and_keeps_the_original_fault( + monkeypatch, tmp_path +): + """A restore that cannot write leaves a HALF-WRITTEN spec, which is the loudest thing + this can be — so it raises through the guard rather than degrading, and the abort + record says `failed` so both surfaces print the "restore it from git" remedy instead + of "the spec was left as the re-arm found it". + + The chain is the other half of the claim. `_restore_rearmed_spec` raises WHILE the + original fault is being handled, so that fault rides in `__context__` and the + operator sees both causes rather than a `RearmError` that has erased the reason the + re-arm aborted in the first place. + + Only the restore's writer is broken: the flip and the strip go through `verify` and + `devcontract`, so this injection cannot pre-empt the writes it is meant to fail to + undo. + + Ablation: move the abort record out of `_rollback_rearm`'s `finally` into its success + path and the `failed` row disappears entirely — the raise still propagates, which is + why the record and not the exception is what grades this. + """ + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + + def no_space(*_a, **_kw): + raise OSError(28, "No space left on device") + + def probe_boom(repo, baseline): + raise MemoryError("not a git answer") + + monkeypatch.setattr(runs.verify, "commits_above", probe_boom) + monkeypatch.setattr(runs, "atomic_write_bytes_confined", no_space) + with pytest.raises(runs.RearmError, match="cannot restore") as excinfo: + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert str(spec) in str(excinfo.value) + chain = [] + exc: BaseException | None = excinfo.value + while exc is not None: + chain.append(exc) + exc = exc.__cause__ or exc.__context__ + assert any(isinstance(e, MemoryError) for e in chain) # the original fault survives + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "failed" + # the record names the fault the re-arm ABORTED on, not the one the rollback hit — + # the second is in the exception the operator already has + assert "MemoryError" in aborted["error"] + + +@pytest.mark.parametrize("append_fault", [TypeError, OSError]) +def test_rearm_keeps_the_original_fault_when_the_abort_record_cannot_be_written( + monkeypatch, tmp_path, append_fault +): + """Writing the abort record is an OBSERVATION, and an observation that cannot be made + must not REPLACE the fault the operator is being told about. + + `_rollback_rearm` journals `rearm-aborted` from a `finally` that runs while the + original fault is unwinding, so anything that append raises escapes in its place — + and the whole re-raise invariant this transaction is built on (the two pinned + `MemoryError` rows) dies quietly with it. The rollback itself has already completed + by then, so nothing about DW-79/DW-83 is at stake in that suppression; only the + breadcrumb is lost, and the operator still receives the fault that explains why. + + BOTH rows matter and they grade different halves. `OSError` is the obvious shape (an + unwritable journal) and passes under either breadth. `TypeError` is the one that + grades the WIDTH: `Journal.append` serializes caller-supplied values and opens a + file, so `json.dumps` and the open can raise outside the filesystem taxonomy + entirely. + + Ablation: narrow the catch back to `except OSError` and the `TypeError` row reddens + with `TypeError` where the `MemoryError` should be, while the `OSError` row stays + green — which is exactly why one row alone is no oracle. + """ + from bmad_loop.journal import Journal + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + + def probe_boom(repo, baseline): + raise MemoryError("not a git answer") + + real_append = Journal.append + + def flaky(self, kind, **fields): + if kind == "rearm-aborted": + raise append_fault("the abort record could not be written") + return real_append(self, kind, **fields) + + monkeypatch.setattr(runs.verify, "commits_above", probe_boom) + monkeypatch.setattr(Journal, "append", flaky) + with pytest.raises(MemoryError, match="not a git answer"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + # the rollback ran BEFORE the record was attempted, so the transaction still held + assert spec.read_bytes() == before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + # ...and the suppressed append left no entry, which the acceptance criterion is + # deliberately conditioned on rather than promising one here + assert not _kinds(run_dir, "rearm-aborted") + + +def test_rearm_lets_an_interrupt_from_the_abort_append_leave(monkeypatch, tmp_path): + """The abort record's append suppresses `Exception` and deliberately NOT + `KeyboardInterrupt` — the ONE place in this transaction where the breadth is narrower + than the guard's own `BaseException`, and the asymmetry has to be graded from the + side the sibling rows cannot reach. + + It is sound only because of WHERE this `finally` runs: the rollback has already + completed by the time the record is attempted, so the spec is back to the bytes the + re-arm found and an interrupt escaping here cannot reproduce DW-79/DW-83. What IS at + stake is the operator's Ctrl-C. Swallowing it to keep a breadcrumb would answer a + stop they issued themselves with a `MemoryError` traceback, and would leave the + process running past the point they asked it to stop. + + Ablation: broaden that catch to `except BaseException` and this reddens with the + `MemoryError` arriving in the interrupt's place. Neither row of + `test_rearm_keeps_the_original_fault_when_the_abort_record_cannot_be_written` + reddens there, because `TypeError` and `OSError` are both already `Exception`. + """ + from bmad_loop.journal import Journal + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + + def probe_boom(repo, baseline): + raise MemoryError("not a git answer") + + real_append = Journal.append + + def interrupted(self, kind, **fields): + if kind == "rearm-aborted": + raise KeyboardInterrupt + return real_append(self, kind, **fields) + + monkeypatch.setattr(runs.verify, "commits_above", probe_boom) + monkeypatch.setattr(Journal, "append", interrupted) + # the INTERRUPT is what leaves, not the fault the re-arm aborted on + with pytest.raises(KeyboardInterrupt): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + # ...and letting it through costs nothing: the rollback ran first, so the spec is as + # the re-arm found it and the story is still armed for a retry + assert spec.read_bytes() == before + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + assert not _kinds(run_dir, "rearm-aborted") + + +def test_rearm_records_unknown_when_the_rollback_cannot_read_the_spec(monkeypatch, tmp_path): + """`unchanged` is earned ONLY by reading the file and proving it byte-equal, so an + undo that could not even LOOK must answer `unknown`. + + This is the read-failure arm, and it is a different arm from the one the sentinel row + grades: there `original` is `None` and `_restore_rearmed_spec` returns before touching + the disk, so that row cannot reach this code at all. Here the preimage was captured + normally and the file is gone by the time the undo runs — another actor removed it + mid-window — so `read_bytes` raises, the undo declines to re-create a file it did not + delete, and it says so. + + Folding that into `unchanged` asserted a byte-equality the producer never checked, and + the surfaces then told the operator the spec "was left exactly as the re-arm found + it" about a file that is not there. + + Ablation: make the `except FileNotFoundError` arm in `_restore_rearmed_spec` return + `"unchanged"` and this reddens on the recorded `rollback` first; drop that assertion + and the rendered message reddens behind it. + """ + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + + def vanishes(repo, baseline): + spec.unlink() # a concurrent actor removes it AFTER the flip published + raise MemoryError("not a git answer") + + monkeypatch.setattr(runs.verify, "commits_above", vanishes) + with pytest.raises(MemoryError, match="not a git answer"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert not spec.exists() # the undo does NOT fight the actor that removed it + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "unknown" + _severity, message, _next_step = runs.rearm_event_notice(aborted) + assert "could not confirm what it left on disk" in message + assert "left exactly as the re-arm found it" not in message + + +def test_rearm_restores_the_spec_when_the_rollbacks_read_only_faults(monkeypatch, tmp_path): + """A read that could not be PERFORMED is not evidence the spec is fine. + + The row above grades the one read fault that ANSWERS something: the file is gone, so + nothing on disk carries the flip and there is nothing to put back. Every other fault + — EIO, EMFILE, a transient EACCES — says nothing about what is on disk, and this read + is only the "already identical, skip the write" shortcut. Answering `unknown` there + abandoned the undo on exactly the runs that still needed it: `save_state` leaves the + story ESCALATED while the spec keeps the re-arm's status flip and its stripped + `## Auto Run Result`, which is the split state this whole transaction exists to + prevent. + + The fault is armed only for the ROLLBACK's read. The preimage capture upstream goes + through the same call, and faulting that instead degrades `original` to `None` and + grades the sentinel arm — a different row entirely, which is why the arming flag is + set from inside the fake that triggers the abort. + + Ablation: fold the two `except` arms back into one `except OSError: return "unknown"` + and this reddens on the spec's bytes first, then on the recorded `rollback`. + """ + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + found = spec.read_bytes() + real_read_bytes = Path.read_bytes + armed: list[bool] = [] + + def only_during_the_rollback(self): + if armed and self == spec: + raise OSError(errno.EIO, "Input/output error") + return real_read_bytes(self) def boom(repo, baseline): + armed.append(True) # the flip and the preimage capture are already behind us raise MemoryError("not a git answer") monkeypatch.setattr(runs.verify, "commits_above", boom) + monkeypatch.setattr(Path, "read_bytes", only_during_the_rollback) with pytest.raises(MemoryError, match="not a git answer"): - runs.rearm_escalation(run_dir, isolated_redrive=False) + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + armed.clear() # let the assertions read the file back + + assert spec.read_bytes() == found # the flip WAS undone, not abandoned + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "restored" + + +def test_rearm_abort_without_a_spec_records_an_empty_locator(monkeypatch, tmp_path): + """A re-arm that never resolved a spec path writes `""` there, NOT the story key. + + `spec_file` is the SPEC's locator on all five `rearm-*` kinds and + `diagnostics._JOURNAL_ALIAS_FIELDS` routes it by that field NAME into the `spec` + namespace. Falling back to the story key would push an identifier through the wrong + namespace — rendered as a spec that does not exist — and the notice would name it as + a file. The empty string routes nowhere and renders as `(none)`. + + Ablation: replace the `""` fallback with `story_key` and both halves redden — the + field carries the key, and the notice names it where `(none)` belongs. + """ + from bmad_loop.model import Phase + + run = escalated_run(tmp_path, "r1", story_key="1-1-a", git_project=True) + run_dir = run.run_dir + + def boom(run_dir_, state_): + raise OSError(28, "No space left on device") + + monkeypatch.setattr(runs, "save_state", boom) + with pytest.raises(OSError, match="No space left on device"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["spec_file"] == "" + assert aborted["rollback"] == "unknown" # no spec path ⇒ no claim about any file + _severity, message, _next_step = runs.rearm_event_notice(aborted) + assert "(none)" in message + assert "1-1-a" not in message + + +def test_rearm_records_failed_when_the_rollback_write_is_interrupted(monkeypatch, tmp_path): + """An interrupt during the restore WRITE leaves the spec in exactly the state `failed` + describes — flipped and not put back — so that is what the record must say. + + `_rollback_rearm`'s inner arm catches `BaseException` for this reason, and the breadth + is as load-bearing as the guard's own: the restore is a file write, and a Ctrl-C + landing in it is the ordinary way for one to be abandoned half-done. Under a narrowed + `except Exception` the interrupt skips the arm, `rollback` keeps its `unknown` floor, + and the `finally` then records "could not confirm what it left on disk" for a spec + the run knows perfectly well it left part-written — the one outcome whose remedy is + not moot. + + Ablation: narrow that inner arm to `except Exception` and this reddens on the + recorded `rollback` (`unknown` for `failed`). The `KeyboardInterrupt` still + propagates either way, which is why the raise alone is no oracle. + """ + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + + def probe_boom(repo, baseline): + raise MemoryError("not a git answer") + + def interrupted(*_a, **_kw): + raise KeyboardInterrupt + + monkeypatch.setattr(runs.verify, "commits_above", probe_boom) + monkeypatch.setattr(runs, "atomic_write_bytes_confined", interrupted) + with pytest.raises(KeyboardInterrupt): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "failed" + # the record still names the fault the RE-ARM aborted on, not the interrupt + assert "MemoryError" in aborted["error"] + _severity, message, next_step = runs.rearm_event_notice(aborted) + assert "may be left part-written" in message + assert next_step.startswith("Restore the spec") + + +def test_rearm_does_not_roll_back_a_commit_that_already_landed(monkeypatch, tmp_path): + """`save_state` commits by ATOMIC REPLACE, so a fault escaping the call does not prove + the transaction failed — and rolling the spec back after a commit that DID land builds + the mirror image of the defect this guard closes. + + The rename is a single instant; the call around it is not. An interrupt delivered + between the replace and the return unwinds through the guard with `state.json` already + describing a PENDING, re-armed task. Undoing the spec there leaves persisted state + re-armed against a spec that is not, and reports it as "nothing was persisted, the + story is still escalated" — a false sentence on both operator surfaces, and a re-drive + that reads the escalated attempt's terminal status on its first save. + + Control flow cannot see this (there is no statement after `save_state` on that path), + so the guard asks the DISK, the only witness of a rename. No `rearm-aborted` record is + written on this leg either: every rendering of that kind asserts nothing was + persisted, and there is no value of `rollback` that is true here. + + Ablation: delete the `_rearm_commit_landed` check from the guard and this reddens on + the flipped-status assertion — the spec is rolled back underneath committed state — + with the abort-record assertion reddening behind it. + """ + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + real_save_state = runs.save_state + + def commits_then_dies(run_dir_, state_): + real_save_state(run_dir_, state_) # the atomic replace LANDS... + raise KeyboardInterrupt # ...and the call is interrupted on its way out + + monkeypatch.setattr(runs, "save_state", commits_then_dies) + with pytest.raises(KeyboardInterrupt): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + text = spec.read_text(encoding="utf-8") + assert "status: ready-for-dev" in text # the flip STANDS beside the commit + assert "## Auto Run Result" not in text + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.PENDING + assert not _kinds(run_dir, "rearm-aborted") + + +@pytest.mark.parametrize("probe_fault", [KeyboardInterrupt, OSError]) +def test_rearm_rolls_back_when_the_commit_probe_itself_fails(monkeypatch, tmp_path, probe_fault): + """`_rearm_commit_landed` is asked a question ON THE ERROR PATH, so a fault raised + ANSWERING it must not cost the rollback. + + Its one call site sits inside the transaction guard's `except BaseException` arm and + runs BEFORE `_rollback_rearm`, so anything escaping the probe escapes the guard too + and the undo never happens — the spec left flipped to the re-drive's status and + stripped of its `## Auto Run Result`, against a task the run still calls ESCALATED, + with no `rearm-aborted` record. That is DW-79/DW-83 reached through the very code + added to prevent its mirror image, which is why the probe degrades to "not committed" + — roll back — on ANY fault rather than only on an `Exception`. + + The probe reads and PARSES a file, so `KeyboardInterrupt` there is an ordinary + outcome, not a contrivance: `load_state` is a `read_text` plus a `json.loads` plus a + `RunState.from_dict`, and an operator's Ctrl-C lands wherever it lands. + + Swallowing that interrupt costs nothing the operator asked for. The rollback is a + repair write whose omission IS the defect, and the guard's `raise` still propagates + the original `MemoryError` a spec-sized write later — which the last assertion pins. + + BOTH rows matter and they grade different halves. `OSError` (a corrupt or unreadable + state file) passes under either breadth. `KeyboardInterrupt` is the one that grades + the WIDTH. + + Ablation: narrow the probe's catch back to `except Exception` and the + `KeyboardInterrupt` row reddens — the spec comes back flipped and no abort record + exists — while the `OSError` row stays green, which is exactly why one row alone is + no oracle. + """ + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + before = spec.read_bytes() + + def probe_boom(repo, baseline): + raise MemoryError("not a git answer") + + real_load_state = runs.load_state + calls = [] + + def unreadable(run_dir_): + # `rearm_escalation` opens with its OWN `load_state`; only the probe's call, + # made from inside the guard arm, is the one under test + calls.append(1) + if len(calls) > 1: + raise probe_fault("the commit probe could not read the state file") + return real_load_state(run_dir_) + + monkeypatch.setattr(runs.verify, "commits_above", probe_boom) + monkeypatch.setattr(runs, "load_state", unreadable) + with pytest.raises(MemoryError, match="not a git answer"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert spec.read_bytes() == before # the rollback ran despite the probe failing + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "restored" + assert "MemoryError" in aborted["error"] + + +def test_rearm_refuses_a_spec_whose_bytes_it_could_not_capture(monkeypatch, tmp_path): + """The preimage read is the transaction's own precondition, so a spec that IS a file + and whose bytes could not be captured must FAIL BEFORE the first write. + + That read is one syscall among many against a file three later writers open + independently, so a TRANSIENT fault (EIO on a network mount, a momentary EACCES, + ENFILE under load) can be followed by writes that all succeed. `spec_before` is then + `None`, the abort further down records `unknown` and puts nothing back, and the + re-arm exits with the flip published against a task still ESCALATED — DW-79/DW-83 + reached through the guard's own preimage. + + The fake fails only the FIRST read of this spec, which is what makes that reachable: + every later read succeeds, so without the refusal the re-arm runs to completion. + + A path that is NOT a file keeps degrading to `None` — a missing spec, a dangling link + and a directory all answer `False` from every writer below, so there is genuinely + nothing to undo, and those shapes stay warn-and-continue. + + Ablation: drop the `is_file()` refusal and this reddens with `DID NOT RAISE` — the + re-arm completes, the spec ends up flipped and stripped, and nothing records it. + """ + from bmad_loop.model import Phase + + run_dir, spec, _patch = _stale_restore_tree(tmp_path) + real_read_bytes = Path.read_bytes + before = real_read_bytes(spec) + failed_once = [] + + def flaky(self): + if self == spec and not failed_once: + failed_once.append(1) + raise OSError(5, "Input/output error") + return real_read_bytes(self) + + monkeypatch.setattr(Path, "read_bytes", flaky) + with pytest.raises(runs.RearmError, match="refuses to write a spec it could not capture"): + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert real_read_bytes(spec) == before # nothing was written, so nothing to undo + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.ESCALATED + (aborted,) = _kinds(run_dir, "rearm-aborted") + assert aborted["rollback"] == "unknown" # no preimage ⇒ no claim about the file + + +def test_ordinary_rearm_writes_no_abort_record(tmp_path): + """The negative side of the transaction: a re-arm that reaches `save_state` must + leave no trace of a rollback that never happened. + + An abort record on a successful re-arm would print "nothing was persisted, the story + is still escalated" beside `re-armed ` on the very same stderr — the + contradiction that trains an operator to stop reading the warnings. + + Ablation: move the `_rollback_rearm(...)` call out of the `except` arm into a + `finally` and this reddens. + """ + from bmad_loop.model import Phase + + run_dir, spec = _escalated_run(tmp_path, _SPEC_WITH_ARR) + + runs.rearm_escalation(run_dir, isolated_redrive=False, resolution_recorded=True) + + assert not _kinds(run_dir, "rearm-aborted") + assert load_state(run_dir).tasks["1-1-a"].phase == Phase.PENDING + assert "## Auto Run Result" not in spec.read_text(encoding="utf-8") + assert "status: ready-for-dev" in spec.read_text(encoding="utf-8") def test_archive_run(tmp_path): @@ -3949,10 +4765,13 @@ def test_task_spec_root_yields_the_project_when_the_worktree_cannot_confine_the_ `relative_to(worktree_path)` raises, so this pair means the spec is lexically outside the mount. `task_spec_path` passes an absolute path through untouched, so answering the worktree here names a root that can NEVER contain the anchored path: - `devcontract._atomic_write_spec` gates on the same lexical `is_relative_to` and - would silently take the plain no-follow arm — losing #593's O_NOFOLLOW walk — while - `_restore_rearmed_spec`, which calls `atomic_write_bytes_confined` directly, would - raise `UnconfinedWriteError` and turn a recoverable re-arm abort into a lost undo. + `devcontract._atomic_write_spec` gates on the same lexical `is_relative_to` and would + silently take the plain no-follow arm, losing #593's O_NOFOLLOW walk — and so would + `_restore_rearmed_spec`, the re-arm's undo, which now selects its writer the same + lexical way rather than calling `atomic_write_bytes_confined` directly. All four + degrade together, which is the point: the undo that once RAISED `UnconfinedWriteError` + here, turning a recoverable re-arm abort into a lost one, is the asymmetry that + parity removed. The project is not guaranteed to contain it either; where nothing does, the write lands on the arm it already took. That is not unconditional, and the exception is diff --git a/tests/test_sprintstatus_advance.py b/tests/test_sprintstatus_advance.py index cce63b8d..62245e64 100644 --- a/tests/test_sprintstatus_advance.py +++ b/tests/test_sprintstatus_advance.py @@ -460,7 +460,7 @@ def boom(path, data: bytes, *, follow_symlinks=True, require_writable_target=Fal @pytest.mark.skipif(sys.platform == "win32", reason="POSIX symlinks") def test_advance_writes_through_a_symlinked_board(tmp_path): """The row that grades this SITE's `follow_symlinks` argument — the DEFAULT - here, unlike the three spec writers, which pass False to match the + here, unlike the spec writers, which pass False to match the name-replacing `atomic_replace` they already had. The default is what preserves behaviour: `write_text` opened through a link, so diff --git a/tests/test_stories_engine.py b/tests/test_stories_engine.py index 664a329d..63eac7f0 100644 --- a/tests/test_stories_engine.py +++ b/tests/test_stories_engine.py @@ -1365,7 +1365,7 @@ def test_blocked_resolve_rearm_then_redispatch_to_done(project): assert not any(s.role == "dev" for s in adapter.sessions) # story 2 not leapfrogged # human fixed the frozen spec → re-arm (must run while still escalation-paused) - runs.rearm_escalation(engine.run_dir, "1", isolated_redrive=False) + runs.rearm_escalation(engine.run_dir, "1", isolated_redrive=False, resolution_recorded=True) assert status_of(read_frontmatter(story_spec(project, "1"))) == "ready-for-dev" # resume re-drives the re-armed story, then continues the schedule to story 2 @@ -1406,7 +1406,7 @@ def test_resolved_wedge_is_still_gated_on_redispatch(project): assert wedged.phase == Phase.ESCALATED and wedged.attempt == 0 and not wedged.sessions runs.rearm_escalation( - engine.run_dir, "1", isolated_redrive=False + engine.run_dir, "1", isolated_redrive=False, resolution_recorded=True ) # human fixed the frozen spec assert load_state(engine.run_dir).tasks["1"].rearmed # ...and the re-drive is armed # a gate on story 1 lands while the run is down @@ -1441,7 +1441,7 @@ def test_sentinel_rearm_deletes_by_recorded_verdict_e2e(project): assert engine.run().paused assert load_state(engine.run_dir).tasks["1"].sentinel_kind == "unresolved" # recorded - runs.rearm_escalation(engine.run_dir, "1", isolated_redrive=False) + runs.rearm_escalation(engine.run_dir, "1", isolated_redrive=False, resolution_recorded=True) assert not sentinel.exists() # cleared by the recorded verdict assert (engine.run_dir / "sentinels" / "1-unresolved.md").is_file() # copy preserved reloaded = load_state(engine.run_dir) diff --git a/tests/test_sweep.py b/tests/test_sweep.py index a75b1a2c..93cf106c 100644 --- a/tests/test_sweep.py +++ b/tests/test_sweep.py @@ -2405,8 +2405,10 @@ def test_triage_session_env_fault_escalates_then_resume_restores_budget(project) # ...and it does so in a NEW generation. The attempt reset above is exactly what # would otherwise re-mint `attempt == 1` — an id byte-equal to the abandoned # attempt's, pointing the fresh record at the abandoned cycle's - # tasks//escalation.json, which `resolve._gather_escalations` reads per record. - # (result.json is not the hazard here: both start_sessions unlink it on launch.) + # tasks//escalation.json. Both adapters now clear cycle outputs at launch, and + # `resolve._gather_escalations` opens each distinct task_id once, but neither makes + # two historical records stop aliasing one mutable directory. The fresh id preserves + # a separate artifact namespace for each cycle, independent of cleanup. assert resumed.state.tasks["sweep-triage"].generation == 1 assert [s.task_id for s in radapter.sessions] == ["sweep-triage-triage-1-g1"] assert radapter.sessions[0].task_id not in abandoned @@ -3465,7 +3467,11 @@ def test_sweep_bundle_restore_redrive_reaches_done_and_clears_latch(project, mon patch.write_text("dummy\n") runs.rearm_escalation( - engine.run_dir, "dw-fix", restore_patch=str(patch), isolated_redrive=False + engine.run_dir, + "dw-fix", + restore_patch=str(patch), + isolated_redrive=False, + resolution_recorded=True, ) resumed, adapter = resume_sweep( @@ -3506,7 +3512,11 @@ def test_sweep_restore_redrive_exhaustion_pauses_not_defers(project, monkeypatch patch.parent.mkdir(parents=True, exist_ok=True) patch.write_text("dummy\n") runs.rearm_escalation( - engine.run_dir, "dw-fix", restore_patch=str(patch), isolated_redrive=False + engine.run_dir, + "dw-fix", + restore_patch=str(patch), + isolated_redrive=False, + resolution_recorded=True, ) resumed, _ = resume_sweep(project, engine, [lambda spec: SessionResult(status="died")]) @@ -3528,7 +3538,7 @@ def test_sweep_from_scratch_redrive_exhaustion_pauses_not_defers(project): ) engine = _run_to_dev_escalation(project, policy=policy) runs.rearm_escalation( - engine.run_dir, "dw-fix", isolated_redrive=False + engine.run_dir, "dw-fix", isolated_redrive=False, resolution_recorded=True ) # from-scratch, no restore resumed, _ = resume_sweep(project, engine, [lambda spec: SessionResult(status="died")]) @@ -4676,7 +4686,9 @@ def test_rearmed_bundle_redrives_when_triage_json_lost(project): # cached triage plan reloaded and re-emitted its name. Recovery now keys on # the persisted task, so losing the cache changes nothing. engine = _run_to_dev_escalation(project) - runs.rearm_escalation(engine.run_dir, "dw-fix", isolated_redrive=False) + runs.rearm_escalation( + engine.run_dir, "dw-fix", isolated_redrive=False, resolution_recorded=True + ) _lose_triage(engine.run_dir) resumed, adapter = resume_sweep(project, engine, _redrive_script(project)) @@ -4698,7 +4710,9 @@ def test_fresh_triage_different_bundle_name_no_double_drive(project, corruption) # would orphan the re-armed one. It must re-drive by identity, and its ids # must have left the open set before the fresh triage sees them. engine = _run_two_bundle_dev_escalation(project) - runs.rearm_escalation(engine.run_dir, "dw-fix", isolated_redrive=False) + runs.rearm_escalation( + engine.run_dir, "dw-fix", isolated_redrive=False, resolution_recorded=True + ) _lose_triage(engine.run_dir, corruption) fresh = triage_result( @@ -4736,7 +4750,11 @@ def test_restore_patch_latch_honored_when_triage_json_lost(project, monkeypatch) patch.parent.mkdir(parents=True, exist_ok=True) patch.write_text("dummy\n") runs.rearm_escalation( - engine.run_dir, "dw-fix", restore_patch=str(patch), isolated_redrive=False + engine.run_dir, + "dw-fix", + restore_patch=str(patch), + isolated_redrive=False, + resolution_recorded=True, ) _lose_triage(engine.run_dir) @@ -4861,7 +4879,9 @@ def test_regenerated_intent_when_bundle_file_missing(project): # The triage session's authored prose is the one unrecoverable piece; the # verbatim ledger entries are re-attached and become the contract. engine = _run_to_dev_escalation(project) - runs.rearm_escalation(engine.run_dir, "dw-fix", isolated_redrive=False) + runs.rearm_escalation( + engine.run_dir, "dw-fix", isolated_redrive=False, resolution_recorded=True + ) _lose_triage(engine.run_dir) intent = Path(engine.state.tasks["dw-fix"].bundle_file) intent.unlink() diff --git a/tests/test_tui_app.py b/tests/test_tui_app.py index 7bfbbc44..1ec73aa5 100644 --- a/tests/test_tui_app.py +++ b/tests/test_tui_app.py @@ -4517,6 +4517,34 @@ async def test_story_checkpoint_card_surfaces_real_review_cycles(project, monkey assert "verification passed" not in line +def test_tui_rearm_refuses_an_alive_run_before_any_mutation(project, monkeypatch): + """The liveness helper's result must control the re-arm, not merely be observed.""" + from bmad_loop import runs + + notes: list[str] = [] + rearms: list[str] = [] + run_id = "20260611-100000-aaaa" + run_dir = project.project / RUNS_DIR / run_id + app = BmadLoopApp(project.project) + + def fail_if_rearm_continues(_path): + raise AssertionError("continued past liveness gate") + + monkeypatch.setattr(data, "liveness", lambda _run_dir: "alive") + monkeypatch.setattr(app, "notify", lambda message, **_kwargs: notes.append(message)) + monkeypatch.setattr(policy_mod, "load", fail_if_rearm_continues) + monkeypatch.setattr( + runs, + "rearm_escalation", + lambda _run_dir, story_key, **_kwargs: rearms.append(story_key), + ) + + app._do_rearm(run_id, run_dir, "1") + + assert rearms == [] + assert notes == [f"run {run_id} may still be live — stop it first"] + + async def test_escalation_rearm_resumes_when_resolution_ready(project, monkeypatch): from bmad_loop import resolve, runs @@ -4550,6 +4578,67 @@ async def test_escalation_rearm_resumes_when_resolution_ready(project, monkeypat await until(pilot, lambda: rearms == ["1"] and calls == ["20260611-100000-aaaa"]) +async def test_tui_rearm_does_not_move_the_escalation_watermark(project, monkeypatch): + """DW-11, on the one re-arm surface a stale `resolution.json` actively invites. + + Every other TUI row here monkeypatches `runs.rearm_escalation` away, so none can + observe what it stamps — this one lets the REAL function run. The marker on disk is + the shape that matters: `resolve.run_session` is the only thing in `src/` that + unlinks it and this gesture never calls it, so the marker survived the CLI cycle + that consumed it, and `resolution_ready` (the sole enabler of this button) still + reads True. `_do_rearm` therefore has to declare `resolution_recorded=False` from + what it KNOWS — it ran no session — rather than from what is on disk, which is + exactly the verdict `_restore_recorded` already records for this surface. + + The watermark is seeded to 1 over a two-record trail so "did not move" is + distinguishable from "was never set"; `generation` is the positive control that the + re-arm really ran. + + Ablation: pass `resolution_recorded=True` from `_do_rearm` (or gate the stamp on + `resolution_path(...).is_file()` inside `rearm_escalation`) and this reddens at + 2 != 1.""" + from bmad_loop import resolve + from bmad_loop.engine import _session_task_id + from bmad_loop.journal import load_state + + monkeypatch.setattr(launch, "mux_available", lambda: True) + monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: None) + monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") + run_dir, _spec = _stories_paused_run( + project.project, + stage="escalation", + spec_status="blocked", + spec_checkpoint=False, + blocked_result="Blocked: needs a human decision on the auth scheme.", + ) + state = load_state(run_dir) + task = state.tasks["1"] + task.phase = Phase.ESCALATED + task.sessions.clear() + for seq in (1, 2): + task.record_session( + SessionRecord( + task_id=_session_task_id("1", "review", seq, 0), role="dev", status="completed" + ) + ) + task.escalations_resolved_upto = 1 # an earlier CLI cycle answered the first record + save_state(run_dir, state) + # the marker that cycle's agent wrote — nothing deleted it at its re-arm + marker = resolve.resolution_path(run_dir, "1") + marker.parent.mkdir(parents=True, exist_ok=True) + marker.write_text("{}", encoding="utf-8") + + app = BmadLoopApp(project.project) + async with app.run_test() as pilot: + await until(pilot, lambda: isinstance(app.screen, DashboardScreen)) + app._do_rearm("20260611-100000-aaaa", run_dir, "1") + await pilot.pause() + + rearmed = load_state(run_dir).tasks["1"] + assert rearmed.escalations_resolved_upto == 1 # NOT len(sessions) == 2 + assert rearmed.generation == 1 # positive control: the re-arm ran + + async def test_escalation_rearm_hands_the_rearm_the_live_isolation_mode(project, monkeypatch): """The mode `runs.rearm_escalation` needs comes from policy.toml, read HERE. @@ -4576,7 +4665,8 @@ async def test_escalation_rearm_hands_the_rearm_the_live_isolation_mode(project, monkeypatch.setattr( runs, "rearm_escalation", - lambda rd, sk, *, isolated_redrive: seen.append(isolated_redrive) or "ready-for-dev", + lambda rd, sk, *, isolated_redrive, resolution_recorded: seen.append(isolated_redrive) + or "ready-for-dev", ) run_dir, _spec = _stories_paused_run( project.project, @@ -4746,7 +4836,7 @@ async def test_escalation_rearm_surfaces_a_failed_baseline_advance(project, monk monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") - def fake_rearm(rd, sk, *, isolated_redrive=False): + def fake_rearm(rd, sk, *, isolated_redrive=False, resolution_recorded=False): Journal(rd).append( "rearm-baseline-advance-failed", story_key=sk, @@ -4806,7 +4896,7 @@ async def test_escalation_rearm_aims_the_code_root_before_it_rearms(project, mon monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") seen: list = [] - def fake_rearm(rd, sk, *, isolated_redrive=False): + def fake_rearm(rd, sk, *, isolated_redrive=False, resolution_recorded=False): seen.append(load_state(rd).code_root) return "ready-for-dev" @@ -4920,7 +5010,9 @@ async def test_escalation_rearm_surfaces_the_kinds_it_used_to_drop(project, monk TUI's own copy of the chain happened to handle. That copy carried `rearm-baseline-*` only and silently dropped the whole - `stale-restore-*` family, including `stale-restore-commits` — the record + `stale-restore-*` family (and would have dropped `rearm-commits-probe-failed`, + the record that says the commits probe could not answer at all), including + `stale-restore-commits` — the record `cli._echo_rearm_events`' docstring calls the one a human must act on, and the one whose whole point is that nothing else will tell them. All of it is warn-only by contract, so a toast is the only place this path can ever show it, @@ -4936,7 +5028,11 @@ async def test_escalation_rearm_surfaces_the_kinds_it_used_to_drop(project, monk already queued behind this toast. Ablation: make `runs.rearm_event_notice` return None for any one of these kinds - and this reddens on that kind's message alone. + and this reddens on that kind's message alone. Drop the remedy + ("restore it from git or from your own copy") from the `rearm-aborted` `failed` + MESSAGE while keeping it in that arm's `next_step` and only the remedy assertion + reddens — which is the point of grading it here rather than on the CLI, where the + dropped half is still printed. """ from bmad_loop import resolve, runs from bmad_loop.journal import Journal @@ -4947,7 +5043,7 @@ async def test_escalation_rearm_surfaces_the_kinds_it_used_to_drop(project, monk monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") - def fake_rearm(rd, sk, *, isolated_redrive=False): + def fake_rearm(rd, sk, *, isolated_redrive=False, resolution_recorded=False): journal = Journal(rd) journal.append( "stale-restore-commits", @@ -4956,6 +5052,16 @@ def fake_rearm(rd, sk, *, isolated_redrive=False): commits=["c1", "c2"], ) journal.append("stale-restore-excluded", story_key=sk, patch="a.patch", files=["new.txt"]) + # The commits record's TWIN: the probe that could not answer at all. It rides + # this walk because the pair is the whole point — the record above is written + # only when the probe answered, so without this one its absence reads as + # "clean" on the surface that resumes in the same gesture (DW-81). + journal.append( + "rearm-commits-probe-failed", + story_key=sk, + old_baseline="e" * 40, + error=f"GitError: git rev-list {'e' * 40}..HEAD failed in /code: fatal", + ) journal.append( "rearm-baseline-restamp-skipped", story_key=sk, @@ -4968,6 +5074,22 @@ def fake_rearm(rd, sk, *, isolated_redrive=False): spec_file="wt/specs/s1.md", status="ready-for-dev", ) + # `rearm-aborted` is journalled by `runs._rollback_rearm` from the transaction + # guard's error path. It rides this walk for its ROUTING, which is what this test + # grades — the rendering path is real on this surface either way, since a genuine + # abort reaches `_do_rearm`'s `finally` (and so this echo) BEFORE its + # `except RearmError` arm returns. The `failed` outcome is the one chosen on + # purpose: it is the single re-arm kind whose imperative is NOT moot here, because + # this path does not go on to resume, and this surface drops `next_step` — so the + # restore-from-git remedy has to survive in the MESSAGE or a TUI operator never + # gets it at all. + journal.append( + "rearm-aborted", + story_key=sk, + spec_file="wt/specs/s1.md", + error="OSError: [Errno 28] No space left on device", + rollback="failed", + ) return "ready-for-dev" monkeypatch.setattr(runs, "rearm_escalation", fake_rearm) @@ -5005,10 +5127,26 @@ def severity_of(fragment: str) -> str: assert severity_of("2 commit(s) sit below the re-drive's new baseline (ffffffffffff..)") == ( "warning" ) + # ...and its twin, the probe that could not answer — advisory, so a toast is the + # only place this surface can ever show it + assert severity_of("could not list the commits above the abandoned attempt's baseline") == ( + "warning" + ) + # the range is carried in the MESSAGE, because this surface drops `next_step` + assert any("git log eeeeeeeeeeee..HEAD" in n[0] for n in notes), notes + assert not any("e" * 40 in n[0] for n in notes), notes assert severity_of("is not a readable file from here") == "warning" assert severity_of("could not be re-opened to `ready-for-dev`") == "warning" # `note` maps onto Textual's own channel name, not through unchanged assert severity_of("excluded the abandoned restore's new files") == "information" + # the abort record, and specifically the half that only the MESSAGE can carry on a + # surface with no `next_step`: without it a TUI operator is told the spec may be + # part-written and given no remedy for it + assert severity_of("may be left part-written") == "warning" + # the WHOLE remedy, not its first three words: the message names a second source + # because an untracked or out-of-checkout spec has no committed copy, and asserting + # only the "from git" prefix passes for a message that never gained the rest + assert any("restore it from git or from your own copy" in n[0] for n in notes), notes # the CLI's trailing imperative is omitted here: the resume is already queued assert not any("before resuming" in n[0] for n in notes), notes assert any("re-armed 1" in n[0] for n in notes) # the ordinary notice still fires @@ -5042,7 +5180,7 @@ async def test_escalation_rearm_holds_the_resume_it_folds_in(project, monkeypatc monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") - def fake_rearm(rd, sk, *, isolated_redrive=False): + def fake_rearm(rd, sk, *, isolated_redrive=False, resolution_recorded=False): Journal(rd).append( "rearm-spec-write-unreachable", story_key=sk, @@ -5114,7 +5252,7 @@ async def test_escalation_rearm_echoes_residue_when_the_rearm_aborts(project, mo monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") - def fake_rearm(rd, sk, *, isolated_redrive=False): + def fake_rearm(rd, sk, *, isolated_redrive=False, resolution_recorded=False): # exactly the real ordering: residue journalled, THEN the abort Journal(rd).append( "stale-restore-commits", story_key=sk, old_baseline="f" * 40, commits=["c1"] @@ -5175,7 +5313,7 @@ async def test_escalation_rearm_survives_a_corrupt_journal(project, monkeypatch) monkeypatch.setattr(launch, "resume_detached", lambda proj, rid: calls.append(rid)) monkeypatch.setattr(data, "liveness", lambda run_dir: "dead") - def fake_rearm(rd, sk, *, isolated_redrive=False): + def fake_rearm(rd, sk, *, isolated_redrive=False, resolution_recorded=False): Journal(rd).append( "stale-restore-commits", story_key=sk, old_baseline="f" * 40, commits=["c1"] )