fix(live-proof): avoid replaying one-shot terminal proofs

[steipete]

What Problem This Solves

Resolves generated terminal Live Verification plans replaying a one-shot proof by placing the same command in both entry and the first run. In PR 1240, entry passed all five assertions, then the replay correctly refused to overwrite the proof output. The trusted PR 1254 producer run also contained a duplicated entry/run command.

Why This Change Was Made

Make the contract explicit where plans are generated: terminal entry executes automatically before typed steps, and every run is a separate execution. One-shot proofs belong either in entry followed by expectations, or after safe setup as exactly one run. Media guidance must not encourage replay merely to obtain a transition.

The base included PR 1256, which silently removed a leading identical run. This restores command-preserving parsing instead. Its cold-build/final-output retention remains intact. Public schema properties/types/required fields/enums/bounds, driver supervision, deadlines, failure gates, and non-overwrite protection are unchanged.

Maintainer Decision and Review Disposition

The maintainer decision is to preserve every declared terminal command, including an immediate run identical to entry. Command identity cannot tell the parser whether repetition is intentional. The planner owns avoiding accidental one-shot replay; supervision and target non-overwrite guards remain authoritative. A malformed duplicate plan may fail rather than have its program silently rewritten.

Resolved: [P1] Remove the unsandboxed Codex proof command. The public recipe no longer grants full host authority. A dedicated constrained generator ignores user/project configuration and execution rules, disables shell/MCP/web/plugin/agent access, and uses a named restricted filesystem profile with no writable paths or child network access. It tests denied reads/writes, verifies actual startup permissions and tool inventory, and requires zero model tool calls before accepting a decision. The latest proof uses two new generations through the final unchanged helper, not old full-access decisions or retrospective re-attestation.

This applies the review's corresponding rank-up move. The helper is explicitly attested for macOS Seatbelt and Codex CLI 0.150.0-alpha.13; other platforms/versions fail closed pending re-attestation. This is not a blanket no-tools claim: the inspected, fingerprinted Code Mode wrappers expose only sandboxed apply_patch; an optional user-message tool has no host filesystem/network authority. The CLI's own authenticated inference transport is separate from model-directed tool access.

User Impact

One-shot terminal proofs can be planned without accidental replay, explicit repeated commands are never silently discarded, and the public reproduction workflow no longer requires unrestricted model execution. Nonzero exits and real timeouts still fail. Release-note context is retained here and in the proof record, following the release-owned changelog disposition in PR 1256.

OpenClaw Bay Impact

Unaffected. Schema v1, verification lifecycle, publication/receipt contracts, and Bay's observer-only ownership do not change.

Documentation Impact

Updated active docs/live-proof.md and the production review prompt. The proof record documents the controlled fixture, constrained generation, real-tmux reproduction, and explicit platform/toolchain limits. The latest normalized receipt supersedes the historical full-access receipt without rewriting its history.

Evidence

• 138 focused tests passed across decision parsing, review prompt policy/context, and live proof. They cover command preservation, report roundtrips, repeat failure, and retained long-output behavior.
• The final constrained generator produced two fresh decisions from the complete production review prompt and schema. Both were parsed unchanged and executed through real tmux with all five success assertions observed.
• Harmless sandbox probes denied an outside-workspace read and a workspace write. Actual session traces verified restricted permissions, no writable paths, restricted child network, the expected tool inventory, and zero tool calls.
• Independent precommit P0/P1 autoreview of the frozen correction returned no findings and judged the patch correct: the original unrestricted-host-authority finding is resolved for the pinned macOS workflow. That review inspected the supplied bundle and dependencies; the parent separately verified both actual final-helper generations and their real-tmux proof. Committed-branch review remains a landing gate.
• Build, helper syntax/lint/format, static/documentation checks, and diff checks passed. The earlier full local suite's diagnosed fixture failures remain disclosed below.

Real Behavior Proof

Claim: the planner describes a one-shot proof once, the constrained generator does not give model-directed tools unrestricted host authority, and parser/driver preserve intentional repetitions and existing failure gates.

Surface: production prompt/schema → actual constrained Codex generation → production parser → production terminal driver/tmux → schema-v1 verification. The generation phase receives a complete synthetic local fixture as data; it does not execute that fixture. The later real-tmux phase uses an allowlisted dependency-free command that exclusively creates an output directory, records invocations, writes stable bytes, and emits five markers.

Environment: local macOS, Node 24.19.0, tmux 3.7c, and the attested Codex CLI. The fast plan runs the proof only in entry; the 34-second delayed plan uses safe setup followed by one run. The helper denies unexpected permissions/tools or trace failures instead of falling back to full access. No driver deadline was raised.

Exact committed head: a08acddf53cc688d4fd38206898fb35023c9c1f6. Fresh fast and delayed real-tmux replays both passed at this committed head using /opt/homebrew/opt/node@24/bin/node (Node 24.19.0), with that Node directory first on PATH. These are replays of the original constrained-generated decisions, not fresh model calls. Fresh prepare bundles recorded this head, and their source hashes, prompt SHA256, and schema SHA256 matched the original generation bundles exactly. Each decision was copied byte-for-byte; its original generation.json provenance was retained at the original path and explicitly referenced by a generation-reuse.json record with reusedDecision: true and freshModelCall: false. The unchanged helper SHA256 is 2b12954b5afe14cb2d451a5457342ca1ad5f29703b2deb5426f162f80692f41e, matching both original generation attestations. Both replays observed all five required named PASS assertions; additional setup/start expectations were allowed and also passed. Each invoked the proof once before direct replay, which exited 17 without changing the result bytes. Duplicate execution, intentional beforeafter repetition, exit-7 failure, and the real 30-second timeout all behaved as required. No timeout or proof contract was changed.

Local exact-head artifacts (SHA256):

• Fast receipt: .artifacts/terminal-proof-planning/safe-head-fast/receipt.json — b8de86e491438253bf6f303c0eebc2905cbeeb7af719c423e02841ad76d3e052. Reuse provenance: .artifacts/terminal-proof-planning/safe-head-fast/generation-reuse.json — 9844946e8b316e17ad548ce5dcf519438df356890f0ba6d4635ba0954264cf17.
• Delayed receipt: .artifacts/terminal-proof-planning/safe-head-delayed/receipt.json — e0aa2508433e27d5b21cab36e169d477570a0fcda90ed953bf490f32273f7af8. Reuse provenance: .artifacts/terminal-proof-planning/safe-head-delayed/generation-reuse.json — 24904d465832968e3dd038da30b204e241c1530cfe7b8f454725505b748cc9e0.

The committed public normalized receipt retains its actual original runtime head and uncommitted source hashes; it was not rewritten to invent a self-referential commit SHA. These new local receipts provide the exact-head binding.

Observed results: each generated plan invoked the one-shot command once and observed all five named assertions. Direct replay exited 17 and preserved the original bytes. A deliberately duplicated entry/run executed twice and failed its overwrite guard. Identical later commands separated by a file-state change observed beforeafter. Exit 7 and a silent 40-second command failed under existing gates. File bytes and invocation traces support these assertions independently of printed PASS strings.

Provenance: the linked normalized receipt binds the new generation's final-helper hash, prompt/schema/decision hashes, effective permissions/tool inventory, and real-tmux outcomes. Final-head replay binds those same generated decisions and unchanged implementation inputs to the committed head above. Raw agent transcripts and private model identifiers are not included.

Limits: controlled local terminal fixture—not target Worker/apply correctness, hosted review execution, all future model generations, or cross-platform generator support. Recording/transcoding/publication were not tested. No live apply/close, workflow pause, or external proof upload was performed.

PR 1254 status investigation

No separate driver status bug was established. Producer and base drivers are identical. Real tmux probes showed a previous-command gate can expire and final capture can later collect successful output while verification correctly stays failed: the requested next command never executed. The late-capture probe deliberately injected latency, establishing possible ordering rather than the producer's actual timing. The original artifact lacks status-sample timestamps; target JSON containing exit:0 cannot erase a timeout. Driver status logic and deadlines remain unchanged.

Validation and landing gates

Current-head CI evidence is available in the PR Checks tab. Landing requires successful CI, CodeQL, and hermetic e2e results attached to a08acddf53cc688d4fd38206898fb35023c9c1f6, plus clean committed-branch autoreview and a current-head/body ClawSweeper review with no unresolved actionable finding. The final maintainer verification checks those live results rather than freezing a transient pending state in this body.

For historical context, the prior runtime head ca48d41487affa905cb775da8a985d1b128b18c3 passed CI, hermetic e2e, and CodeQL. Those runs do not substitute for checks on the new head.

The earlier full local pnpm run check finished with 3,712 passed, 49 failed, nine skipped on Node 24.19.0; it is not represented as green. Independent diagnosis reproduced 42 inherited Git-pruning failures deleting fixture origin/main, six unavailable pinned-pnpm versions in fresh offline mirrors, and one timing-sensitive setup-budget assertion. Ten representative Git-affected tests passed with process-local pruning disabled; the timing test passed three focused reruns. Owners/fixtures were byte-identical to the base and failed outside the changed parser path. Not every blocked scenario was rerun after removing its host-config blocker. These issues remain a separate follow-up; no global configuration or unrelated tests were changed.

The maintainer has explicitly authorized landing after the above gates pass. This does not authorize live apply/close or workflow mutation.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed August 27, 2026, 3:17 AM ET / 07:17 UTC.

ClawSweeper review

What this changes

The PR preserves every declared terminal proof command, updates planner guidance to avoid accidental one-shot replays, and adds a constrained macOS proof generator with parser and driver coverage.

Regression provenance

Possible regression — probable (reviewed change; known regression link). No predecessor PR is attributed.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

Keep open: the current patch is correct and resolves the prior security finding with an attested constrained generation path; no contributor-facing defect remains in the reviewed diff.

Priority: P2
Reviewed head: a08acddf53cc688d4fd38206898fb35023c9c1f6

Review scores

Measure	Result	What it means
Overall readiness	🦞 diamond lobster (5/6)	Strong constrained runtime evidence and focused parser coverage support the documented behavioral change.
Proof confidence	🦞 diamond lobster (5/6) ✨ media proof bonus	Sufficient (linked_artifact): The PR body and committed receipt provide after-fix constrained Codex generation plus real-tmux execution evidence for both one-shot and intentional-repeat terminal plans.
Patch quality	🦞 diamond lobster (5/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (linked_artifact): The PR body and committed receipt provide after-fix constrained Codex generation plus real-tmux execution evidence for both one-shot and intentional-repeat terminal plans.
Evidence reviewed	6 items	Command-preserving parser: The parser now maps and retains every typed step; the prior leading entry/run normalization is removed, while recommended plans still require an entry and at least one typed step. Planner contract is aligned: The production prompt, schema descriptions, and public live-proof documentation consistently direct one-shot proofs to use either entry plus expectations or setup plus one run. Constrained proof boundary: The new generator pins the inspected CLI version, disables ambient configuration and model-directed shell/network access, verifies the effective restricted profile and tool inventory, and rejects any model tool call.
Findings	None	None.
Security	None	None.

Live Verification

Command: node --test --test-name-pattern='preserves every terminal command including exact entry repeats|terminal entry with expectations executes once' test/decision-parser.test.ts test/live-proof.test.ts

Result: FAIL (failed) — step 1 expect_output preserves every terminal command including exact entry repeats: expected terminal output was not visible within 30 seconds: "preserves every terminal command including exact entry repeats"

node:internal/modules/esm/resolve:271
    throw new ERR_MODULE_NOT_FOUND(
          ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/tmp/clawsweeper-live-proof-1259-0IfRSt/target/dist/clawsweeper.js' imported from /tmp/clawsweeper-live-proof-
1259-0IfRSt/target/test/decision-parser.test.ts
    at finalizeResolution (node:internal/modules/esm/resolve:271:11)
    at moduleResolve (node:internal/modules/esm/resolve:865:10)
    at defaultResolve (node:internal/modules/esm/resolve:992:11)
    at #cachedDefaultResolve (node:internal/modules/esm/loader:701:20)
    at #resolveAndMaybeBlockOnLoaderThread (node:internal/modules/esm/loader:721:38)
    at ModuleLoader.resolveSync (node:internal/modules/esm/loader:759:56)
    at #resolve (node:internal/modules/esm/loader:683:17)
    at ModuleLoader.getOrCreateModuleJob (node:internal/modules/esm/loader:603:35)
    at ModuleJob.syncLink (node:internal/modules/esm/module_job:163:33)
    at ModuleJob.link (node:internal/modules/esm/module_job:253:17) {
  code: 'ERR_MODULE_NOT_FOUND',
  url: 'file:///tmp/clawsweeper-live-proof-1259-0IfRSt/target/dist/clawsweeper.js'
}

Node.js v24.19.0
✖ test/decision-parser.test.ts (93.089984ms)
node:internal/modules/esm/resolve:271
    throw new ERR_MODULE_NOT_FOUND(
          ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/tmp/clawsweeper-live-proof-1259-0IfRSt/target/dist/clawsweeper.js' imported from /tmp/clawsweeper-live-proof-
1259-0IfRSt/target/test/live-proof.test.ts
    at finalizeResolution (node:internal/modules/esm/resolve:271:11)
    at moduleResolve (node:internal/modules/esm/resolve:865:10)
    at defaultResolve (node:internal/modules/esm/resolve:992:11)
    at #cachedDefaultResolve (node:internal/modules/esm/loader:701:20)
    at #resolveAndMaybeBlockOnLoaderThread (node:internal/modules/esm/loader:721:38)
    at ModuleLoader.resolveSync (node:internal/modules/esm/loader:759:56)
    at #resolve (node:internal/modules/esm/loader:683:17)
    at ModuleLoader.getOrCreateModuleJob (node:internal/modules/esm/loader:603:35)
    at ModuleJob.syncLink (node:internal/modules/esm/module_job:163:33)
    at ModuleJob.link (node:internal/modules/esm/module_job:253:17) {
  code: 'ERR_MODULE_NOT_FOUND',
  url: 'file:///tmp/clawsweeper-live-proof-1259-0IfRSt/target/dist/clawsweeper.js'
}

Node.js v24.19.0
✖ test/live-proof.test.ts (97.217891ms)
ℹ tests 2
ℹ suites 0
ℹ pass 0
ℹ fail 2
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 107.425404

✖ failing tests:

test at test/decision-parser.test.ts:1:1
✖ test/decision-parser.test.ts (93.089984ms)
  'test failed'

test at test/live-proof.test.ts:1:1
✖ test/live-proof.test.ts (97.217891ms)
  'test failed'

Assertions:

• FAIL expect_output: preserves every terminal command including exact entry repeats
• FAIL expect_output: terminal entry with expectations executes once

How this fits together

Live Proof turns a Codex review decision into a supervised browser or terminal verification run and then publishes a normalized receipt. This change affects the terminal-plan parser and the instructions that produce its entry command and subsequent steps.

flowchart LR
  A[Codex review decision] --> B[Live Proof plan parser]
  B --> C[Automatic terminal entry]
  C --> D[Typed terminal steps]
  D --> E[Supervised terminal driver]
  E --> F[Verification receipt]

Loading

Before merge

• Resolve merge risk (P1) - A previously generated terminal plan that still repeats its entry as the first run will now execute twice and can correctly trip a non-overwrite guard instead of being silently rewritten.
• Resolve merge risk (P1) - The constrained-generation attestation is intentionally limited to macOS Seatbelt and Codex CLI 0.150.0-alpha.13; use on another platform or CLI version must re-attest rather than inherit this proof.
• Complete next step (P2) - No narrow repair remains; the resolved prior finding leaves routine current-head status verification and normal maintainer landing review.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Patch composition	12 files; +1,266 / -37 lines	The runtime parser is reduced by seven lines; most of the patch is focused proof harness and evidence material.

Root-cause cluster

Relationship: canonical
Canonical: #1259
Summary: This PR is the active corrective work for terminal-plan duplicate handling; PR 1256 is a directly related earlier implementation with retained, distinct output-retention work.

Members:

• partial_overlap: fix(live-proof): keep final results visible after cold builds #1256 - Its merged leading-duplicate normalization is deliberately replaced here, while its cold-build final-output behavior remains intact.
• adjacent_distinct: fix(dashboard): normalize exact-review queue failures #1240 - Its discussion supplies an observed one-shot replay example but its merged change owns a separate dashboard queue failure contract.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

1. Land the documented command-preservation contract (recommended)
   Accept the intentional automation behavior that malformed duplicate plans fail visibly while the updated planner avoids generating accidental one-shot replays.
2. Retain fail-closed attestation scope
   Require a fresh restricted-profile attestation before using the proof generator with another Codex CLI version or platform.

Technical review

Best possible solution:

Keep one explicit command-preserving contract across the parser, schema, prompt, and documentation, with constrained generation remaining fail-closed outside its attested platform and CLI version.

Do we have a high-confidence way to reproduce the issue?

Yes: current source shows the prior leading entry/run normalization, and the committed controlled real-tmux receipt exercises the restored repeated-command and one-shot paths.

Is this the best way to solve the issue?

Yes: preserving declared commands avoids silently rewriting a plan, while explicit generator guidance prevents accidental one-shot replays before execution.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against ff813ac2fa76.

Labels

Label changes:

• add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): The PR body and committed receipt provide after-fix constrained Codex generation plus real-tmux execution evidence for both one-shot and intentional-repeat terminal plans.
• remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.
• remove rating: 🦐 gold shrimp: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

• P2: This is a bounded review-automation reliability correction with limited blast radius.
• merge-risk: 🚨 automation: Terminal plans with an accidental entry/run duplicate now fail visibly rather than being silently normalized.
• merge-risk: 🚨 security-boundary: The PR adds a Codex inference proof helper whose safety depends on its pinned, runtime-attested restricted profile.
• rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): The PR body and committed receipt provide after-fix constrained Codex generation plus real-tmux execution evidence for both one-shot and intentional-repeat terminal plans.
• proof: sufficient: Contributor real behavior proof is sufficient. The PR body and committed receipt provide after-fix constrained Codex generation plus real-tmux execution evidence for both one-shot and intentional-repeat terminal plans.

Evidence

What I checked:

• Command-preserving parser: The parser now maps and retains every typed step; the prior leading entry/run normalization is removed, while recommended plans still require an entry and at least one typed step. (src/clawsweeper-decision-parser.ts:653, a08acddf53cc)
• Planner contract is aligned: The production prompt, schema descriptions, and public live-proof documentation consistently direct one-shot proofs to use either entry plus expectations or setup plus one run. (prompts/review-item.md:510, a08acddf53cc)
• Constrained proof boundary: The new generator pins the inspected CLI version, disables ambient configuration and model-directed shell/network access, verifies the effective restricted profile and tool inventory, and rejects any model tool call. (scripts/e2e/terminal-proof-generate.mjs:59, a08acddf53cc)
• After-fix runtime evidence: The committed receipt records two constrained generations and real-tmux runs: one one-shot execution per generated plan, intentional duplicate failure, intentional later repeat, nonzero-exit failure, and the existing timeout gate. (docs/proof/terminal-proof-planning/receipt.json:125, a08acddf53cc)
• Clean current-main merge: A three-way merge simulation using the PR base, current main, and the PR head reports the changed files as merged without conflicts. (src/clawsweeper-decision-parser.ts:653, a08acddf53cc)
• Feature-history provenance: The relevant parser history includes the current corrective commit, the merged leading-duplicate normalization from PR 1256, and earlier Live Proof work by Peter Steinberger. (src/clawsweeper-decision-parser.ts:653, f211e21fb89d)

Likely related people:

• steipete: Introduced earlier Live Proof planning work and authored the current corrective change. (role: live-proof feature author and recent area contributor; confidence: high; commits: 44eb7ed2e43d, 69adf7592803, a08acddf53cc; files: src/clawsweeper-decision-parser.ts, prompts/review-item.md, scripts/e2e/terminal-proof-generate.mjs)
• vincentkoc: Merged PR 1256, whose leading-terminal-command normalization this PR deliberately replaces while retaining its output-retention work. (role: recent adjacent contributor; confidence: high; commits: f211e21fb89d; files: src/clawsweeper-decision-parser.ts, test/decision-parser.test.ts, test/live-proof.test.ts)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (3 earlier review cycles)

• reviewed 2026-08-27T05:34:55.954Z sha ca48d41 :: needs maintainer review before merge. :: none
• reviewed 2026-08-27T05:46:56.700Z sha ca48d41 :: needs maintainer review before merge. :: none
• reviewed 2026-08-27T06:28:49.206Z sha ca48d41 :: found issues before merge. :: [P1] Remove the unsandboxed Codex proof command

[steipete]

@clawsweeper re-review

The main PR body now records the maintainer decision to preserve every declared terminal command, including immediate duplicates, and resolves the applicable rank-up move. Exact-head CI is green, fresh real-tmux fast/delayed replays passed, and committed-branch autoreview is clean. The separately diagnosed local fixture failures remain disclosed. Please review the current head and body; no repair, autofix, or automerge is requested.

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/33046095706
• Updated: 2026-08-27T06:30:47.929Z