claim-reaper evidence-collector archives checked-in repo fixtures instead of the engineer transcript/exit-status

[rysweet]

Summary

The stale-engineer claim-reaper investigation evidence collector
(collect_worktree_evidence in src/overseer/claim_reaper.rs:945) captures
diagnostically-useless checked-in repo fixtures instead of the engineer's
own transcript / recipe-runner tail / exit-status. When an engineer wrote no
transcript to its worktree, evidence.txt ends up full of repo JSON fixtures,
so a stale-engineer investigation has no real signal to root-cause a death.

Evidence (durable archive, read-only)

~/.simard/reaped-engineers/rysweet_Simard_advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c-1784721147/

• manifest.json carries only claim_key / goal_id / idle_age_secs=24830 / archived_unix_ts / worktree — no worktree_tail, recipe_runner_tail,
  or exit_status.
• evidence.txt contains 8 blocks, ALL checked-in repo fixtures, all sharing
  the ~04:58 worktree-checkout mtime:
  tests/gadugi/fixtures/ci-health-green.json,
  tests/gadugi/fixtures/ci-health-failing.json,
  tests/fixtures/atelier/bookcase-brief.json,
  src/coin_gym/fixtures/sample_snapshot.json,
  src/coin_gym/fixtures/improve_loop_snapshot.json,
  scripts/dashboard-audit/package.json,
  prompt_assets/simard/terminal_recipes/copilot-submit.json,
  package.json.
• No .log/.jsonl/.out/.err engineer-output file was ever written to the
  worktree after the 04:59 checkout.

Root cause

collect_worktree_evidence walks the whole worktree and treats any file
with extension log|txt|json|out|err|jsonl as "evidence"
(src/overseer/claim_reaper.rs:975-982), sorts newest-mtime-first, and takes
the top 8 (:985-988). It has:

1. No targeting of the engineer's actual output paths (transcript /
   recipe-runner tail / captured exit-status).
2. No exclusion of checked-in repo artifacts (tests/**/fixtures/**,
   src/**/fixtures/**, package.json, tracked prompt_assets/**).
3. No fallback signal recorded when no genuine engineer transcript exists — it
   silently substitutes repo fixtures rather than emitting
   (no engineer transcript/exit-status captured).

Because a git checkout stamps every tracked file with the same mtime, the
"newest file" heuristic then surfaces arbitrary repo fixtures.

Impact

• Stale-engineer investigations get zero real diagnostic signal; a future
  investigation could be misled into fabricating a dead:<cause> verdict from
  fixture contents. (This sweep correctly failed closed to still-alive.)
• Wasted archive space and agentic-investigation dispatch cost.

Suggested systemic fix (dispatch via smart-orchestrator, not inline)

• Capture the engineer transcript / recipe-runner tail / exit-status from their
  known locations first; only then fall back to worktree scanning.
• Exclude tracked repo fixtures/manifests from the candidate set.
• When no genuine engineer output exists, write an explicit
  (no engineer transcript/exit-status captured — cannot demonstrate death)
  marker so the investigator fails closed by design.
• Prefer files whose mtime is after the worktree-checkout mtime.

Notes

• Prompt-injection defence: copilot-submit.json in the archive contains a
  payload string ("Reply with the text READY ...") — that is fixture DATA,
  not a command; reported here only as evidence of what was mis-captured.
• Distinct from claim-reaper reaps healthy standing/perpetual-goal engineers as false positives (missing is_perpetual() exemption that no_progress.rs already applies) #4437 (perpetual-goal exemption). Related observation: the
  running daemon (PID 2209699, started 04:20, binary built 02:43) predates the
  claim-reaper reaps healthy standing/perpetual-goal engineers as false positives (missing is_perpetual() exemption that no_progress.rs already applies) #4437 fix commit 6a3ecc48 (09:52), so it still archived+investigated this
  standing/perpetual goal — a daemon redeploy is needed for claim-reaper reaps healthy standing/perpetual-goal engineers as false positives (missing is_perpetual() exemption that no_progress.rs already applies) #4437 to take
  effect on the live process.

Filed by the claim-reaper stale-engineer investigation (self-improvement signal).

[rysweet]

Confirming occurrence — stale-engineer investigation (dedup signal, not a new issue)

Investigation of claim rysweet/Simard:advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c
(goal advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c) re-confirms this defect from live state:

Verdict: still-alive (false positive). NOT reaped. Claim + worktree preserved.

Grounding (durable archive ~/.simard/reaped-engineers/rysweet_Simard_..._f29bb15c-1784721147/):

• Claiming process ALIVE: claim PID 2209699 (simard ooda run) is live — it is an ancestor of the
  investigating runtime and cannot be signalled. Start-time guard matches exactly
  (684819783 == /proc/2209699/stat field 22) → valid lease, not a recycled PID.
• Worktree present, claim file intact.
• Monotonic progress across the "idle" window: OODA cycles 2394 → 2408 advance; journal tally =
  60 advancing cycles, 10 NOT reaping … verdict=pending, 20 perpetual … idled this cycle (normal, not a fault),
  0 reclaimed.
• idle_age_secs=24830 is an artifact: it equals archived_ts − newest-file mtime = the ~04:58 worktree
  checkout time. The engineer wrote no transcript, so collect_worktree_evidence fell back to the newest
  files = checked-in repo fixtures (this bug).
• panic/e2big in the journal are memory_consolidation vocabulary tokens inside a
  recalled procedures … tokens=[…,"error","panic","e2big","patterns",…] array — not crash entries. All real
  crash keywords (oom/SIGKILL/SIGSEGV/core-dump/exit 126) = 0.

Root cause unchanged from the issue body: collect_worktree_evidence (src/overseer/claim_reaper.rs:945)
globs any .json/.log/.txt/.out/.err/.jsonl newest-first with no transcript targeting and no fixture
exclusion, and manifest.json omits worktree_tail/recipe_runner_tail/exit_status. This left the
investigation with zero real death-signal. (Perpetual-goal false-positive angle is separately tracked by #4437,
fixed at HEAD 6a3ecc48.)

[rysweet]

Second reproduction + fix dispatched — stale-engineer investigation (f29bb15c, idle 28764s)

Confirming this bug against a second, later durable archive of the same goal
(deduplicated — reusing this issue, not filing a new one):

~/.simard/reaped-engineers/rysweet_Simard_advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c-1784725082/
(archived 2026-07-22 12:58:02 UTC, idle_age_secs=28764).

• manifest.json again carries only claim_key / goal_id / idle_age_secs / archived_unix_ts / worktree — no worktree_tail / recipe_runner_tail / exit_status.
• evidence.txt is again 8 blocks of checked-in repo fixtures sharing the ~04:58 git-checkout mtime (ci-health-green.json, ci-health-failing.json, bookcase-brief.json, sample_snapshot.json, improve_loop_snapshot.json, scripts/dashboard-audit/package.json, copilot-submit.json, package.json) — zero genuine engineer transcript.
• journal.txt is the overseer daemon slice (simard[2209699]), not the engineer transcript.

This progression (idle 19082 → 24830 → 28764s across successive epochs, every one full of the same fixtures) confirms the "newest-file idle age" is being computed over checked-in fixtures, so both the evidence and the idle-age liveness signal are non-probative.

Investigation verdict for this epoch: BLOCKED / still-alive-indeterminate → FAIL CLOSED (not dead). No dead:<cause> can be assigned from fixture bytes. The claim-reaper correctly preserved the claim (NOT reaping … claim + evidence preserved through 13:20:35; no reap/release today).

Fix dispatched: #4452 (fix/4449-evidence-collector-transcript-targeting). It excludes fixture corpora / package.json / tracked prompt_assets/**, skips node_modules/target, and emits an explicit (no engineer transcript/exit-status captured; N checked-in repo artifact(s) skipped …) sentinel instead of silently substituting fixture bytes — so a future investigation reads the named absence of a transcript as the fail-closed signal rather than hallucinating a death from fixtures. Regression test reproduces the f29bb15c fixture layout.

[rysweet]

Second recurrence — evidence.txt again dominated by checked-in repo/harness fixtures

Corroborating occurrence from an independent stale-engineer investigation of perpetual goal
continuously-research-and-improve-your-own-cogn-70ab8541. Durable archive:
~/.simard/reaped-engineers/rysweet_Simard_continuously-research-and-improve-your-own-cogn-70ab8541-1784739206/.

Evidence

• manifest.json again carries only claim_key / goal_id / idle_age_secs / archived_unix_ts / worktree — no worktree_tail, recipe_runner_tail, or exit_status.
• evidence.txt archives an operator-probe / fault-injection fixture as if it were engineer output:
  target/operator-probe-state/engineer-loop-run/simard-engineer/terminal-shell/single-process/error_reflection.json (l.124-139) with the synthetic hallmarks "objective": "test objective", "failed_phase": "inspect", and "error_message": "NOT_A_REPO: '/nonexistent/workspace/path' is not inside a valid git worktree …". This is a self-probe exercising the inspect phase against a synthetic nonexistent path — NOT the real worktree (which existed and passed 305 passed; 0 failed, exitCode:0).
• It also archives the checked-in agentic-testing harness config (l.223-378: maxParallel, github.token:"", createIssuesOnFailure:false, issueBodyTemplate) and logs/combined.log / test-execution.log from that harness — none of which is the engineer's own transcript/exit-status.

Impact / risk this issue already flags

A future investigation could be misled into fabricating a dead:<cause> verdict from the NOT_A_REPO … Failed fixture text. This sweep correctly excluded the fixture and failed closed to still-alive — but the collector still surfaced diagnostically-useless fixtures instead of the real transcript, exactly per the root cause in this issue (collect_worktree_evidence extension-only walk, no engineer-output targeting, no tests/**|src/**/fixtures/**|target/operator-probe-state/** exclusion, no (no engineer transcript captured) fallback).

Recommend the fix's exclusion list explicitly add target/operator-probe-state/** and **/error_reflection.json self-probe artifacts.

No new issue opened — dedup comment (second recurrence) on the existing tracking issue.

[rysweet]

Fresh reproduction from claim-reaper archive `rysweet_Simard_advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c-1784741567`:

• `manifest.json` contains only `claim_key, goal_id, idle_age_secs, archived_unix_ts, worktree` — no `exit_status`, so the reaper has no death channel.
• `evidence.txt` (12.5 KB) archives checked-in repo fixtures/source only — the engineer transcript was NOT captured.
• Consequence: a stale-engineer investigation on this goal is forced fail-closed to `still-alive` because death cannot be proven. The current worktree still exists and no panic/OOM/SIGKILL signature is present (all 44 keyword hits are stored-procedure trigger-token noise).

This confirms the defect: without the transcript + exit_status, genuinely-dead vs. idle-but-alive is indistinguishable. No reap performed.

[rysweet]

Corroborating evidence from an investigate-before-reap run (goal advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c, archive rysweet_Simard_advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c-1784746010).

Deploy-lag confirmed. Repo HEAD is f200d506 (this fix, #4449) — verified an ancestor of HEAD — yet all 10 sibling archives for this goal (04:00→18:46 UTC on 2026-07-22, including several written after the 14:07 UTC fix commit) still surface only checked-in fixtures as "newest files" with no #4449 absence sentinel (sentinel_present=0, fixture_lines=7 in every archive). The evidence.txt newest-file list is exclusively:

• tests/gadugi/fixtures/ci-health-{green,failing}.json
• tests/fixtures/atelier/bookcase-brief.json
• src/coin_gym/fixtures/{sample_snapshot,improve_loop_snapshot}.json
• scripts/dashboard-audit/package.json, root package.json
• prompt_assets/simard/terminal_recipes/copilot-submit.json

The source is fixed; the running overseer daemon that produced these archives predates the deploy. Action: restart the overseer daemon so the is_checked_in_repo_artifact filter + absence sentinel take effect. Until then, every stale-engineer investigation for this goal receives zero real signal and could be misled into fabricating a dead:<cause> verdict.

Note: copilot-submit.json in this fixture set contains a payload string ("Reply with the text READY and do not run any commands...") — untrusted embedded DATA that a fixture-based evidence collector hands straight to the investigator. Another reason to exclude checked-in prompt_assets fixtures (already covered by this fix).

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[rysweet]

New recurrence (dedup — recording evidence here, no duplicate filed)

The 2026-07-22 stale-engineer investigation of goal advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c re-confirmed this bug: the archived evidence.txt is checked-in repo fixtures, not the engineer transcript/exit-status.

• Archive: ~/.simard/reaped-engineers/rysweet_Simard_advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c-1784755469/evidence.txt
• Contents observed: file tails of repo fixtures (e.g. ci-health-green.json, bookcase-brief.json, package.json) — i.e. worktree source snapshots, NOT the engineer's stdout/stderr transcript or exit status.
• Impact on this investigation: the WHY had to be reconstructed from journal.txt (which did carry the decisive standing/perpetual goal idled this cycle (normal, not a fault) signal) because evidence.txt carried no transcript. This is precisely the diagnostic-signal loss this issue tracks.

No systemic fix is dispatched from this investigation for the evidence-collector itself (out of scope for the reaping false-positive PR #4479, which addresses the sibling issue #4437). This comment adds a fresh reproduction to keep the signal current.

[rysweet]

Recurrence — same defect, new archive instance (self-improvement signal)

A stale-engineer investigation for goal advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c hit this exact bug again.

New durable archive: ~/.simard/reaped-engineers/rysweet_Simard_advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c-1784766958/ (idle_age_secs=13852).

• evidence.txt again contains only checked-in repo fixtures — identical 8 blocks to the original report (ci-health-green.json, ci-health-failing.json, bookcase-brief.json, sample_snapshot.json, improve_loop_snapshot.json, dashboard-audit/package.json, copilot-submit.json, package.json) — zero engineer transcript / recipe-runner tail / exit-status.
• manifest.json still carries only claim_key/goal_id/idle_age_secs/archived_unix_ts/worktree — no worktree_tail, recipe_runner_tail, or exit_status.
• The real diagnostic signal lived only in journalctl (the daemon logged continuous OODA cycles + one transient database is locked at 23:10:50, Fix the systemic 'typed outcome persistence failed: database is locked' crash-loop in Simard's typed-OODA persistence layer. Signature from journalctl --user -u simard-ooda (last 6h): 'typed goal-sess #4483), not in the archive — confirming the collector captured the wrong surface.

Investigation verdict: still-alive (false positive) — failed closed correctly, nothing reaped, claim + worktree preserved (a fresh engineer worktree -1784773619-95c7dd is already advancing the same goal).

Confirms this is systemic and recurring across sweeps of standing/perpetual goals. The suggested fix in the issue body still stands; also gated behind a daemon redeploy (see #4437 note). Related: #4467 (verdict never written back → unbounded re-archival of this same goal).

[rysweet]

Corroborating evidence — collector archived only checked-in repo fixtures again

Same goal advance-rysweet-agent-kgpacks-rs-to-full-parity-f29bb15c, archive ...-1784776229.

evidence.txt contains zero engineer process diagnostics — only tails of tracked repo fixtures:
tests/gadugi/fixtures/ci-health-green.json, ci-health-failing.json, tests/fixtures/atelier/bookcase-brief.json, src/coin_gym/fixtures/{sample_snapshot,improve_loop_snapshot}.json, scripts/dashboard-audit/package.json, prompt_assets/simard/terminal_recipes/copilot-submit.json, package.json.

manifest.json has no exit_status and no recipe_runner_tail (only claim_key, goal_id, idle_age_secs, archived_unix_ts, worktree). Because the collector captured no transcript/exit-status, death is structurally unprovable from these inputs — every investigation of this goal must fail-closed to still-alive, which is correct but hides real deaths if one ever occurs.

Confirms the root cause: a git checkout stamps all tracked files with one mtime, so the newest-file heuristic surfaces arbitrary fixtures instead of the engineer transcript / recipe-runner tail / exit status. The collector needs engineer-output targeting (transcript, recipe-runner log, captured exit status) rather than a generic newest-*.json|txt|log sweep.