Skip to content

fix(spawn): refuse a worktree slot another task's metadata still claims - #1443

Open
leecalcote wants to merge 1 commit into
kunchenguid:mainfrom
leecalcote:fm/fm-worktree-realloc-unlanded
Open

fix(spawn): refuse a worktree slot another task's metadata still claims#1443
leecalcote wants to merge 1 commit into
kunchenguid:mainfrom
leecalcote:fm/fm-worktree-realloc-unlanded

Conversation

@leecalcote

Copy link
Copy Markdown

Symptom

fm-spawn.sh can be handed a worktree pool slot that another in-flight task still owns, launch a new agent on top of it, and silently destroy the first task's unlanded work. Observed 2026-07-21: treehouse status reported the pool as effectively empty while six populated, mid-flight slots existed, and a new spawn reset an in-flight task's worktree to a detached HEAD at the pool's clean base.

Root cause

Worktree availability is decided from live-agent presence, not recorded task ownership. After treehouse get (or orca worktree create) resolves a worktree, validate_spawn_worktree only checks that the path is a real git worktree distinct from the primary checkout - it never checks whether any state/<id>.meta still records that path as an owned task's worktree. So once the pool hands back a slot whose owner process has died (host reboot, the treehouse get subshell exiting while the agent persists, PID/start-time drift), firstmate launches on it unconditionally. state/<id>.meta ownership outlives the agent process, so firstmate holds durable ownership the pool does not.

Fix

  • bin/fm-spawn.sh: add worktree_meta_claimant and enforce it in validate_spawn_worktree. The spawn now refuses to launch onto a resolved worktree that any other task's state/<id>.meta still claims, independent of whether an agent is live in it. The refusal names the owning task so it can be reconciled or torn down. Physical (realpath) paths are compared so a symlinked or relative prefix cannot hide a real collision, and the spawn's own task id is skipped so a recovery re-spawn can reclaim its own recorded slot. Living in validate_spawn_worktree covers treehouse and Orca (and any future backend) at the single chokepoint.
  • The refusal is deliberately loud (stop) rather than silently selecting another slot, so the collision surfaces to the operator.

Note on the pool-side cause

The false-empty pool report itself is a data-loss hazard in the external kunchenguid/treehouse tool, whose availability model has no notion of unlanded git state. This change is the durable firstmate-side guard that fails closed regardless; a companion fix is being filed against kunchenguid/treehouse so that get, when reclaiming a slot whose owner_pid is dead, refuses to destructively reset a worktree that is dirty or ahead of its base.

Verification

  • tests/fm-spawn-worktree-claim-guard.test.sh (new): drives the real fm-spawn.sh and asserts a meta-claimed slot is refused (exit 1, names the owner, leaves the claimant's meta intact, writes no new meta) and that a claimant owning a different worktree does not false-refuse a free slot.
  • tests/fm-spawn-dispatch-profile.test.sh: updated so distinct tasks get distinct worktrees (as a real pool hands out), since two tasks sharing one fake slot now correctly trips the guard; includes a small opt-in per-window worktree map in its fake tmux for the batch case.
  • bin/fm-test-run.sh: the new test is classified in the backend-dispatch family; --check-coverage balances.
  • docs/architecture.md: documents the ownership refusal.
  • bin/fm-lint.sh passes; the changed-surface tests pass.

Closes #1441

Updates from git push no-mistakes

The worktree pool decides slot reuse from live-agent presence, not recorded
task ownership. A crash- or reboot-orphaned but still-owned slot reads as
"empty" (observed 2026-07-21, when treehouse reported the pool empty while six
populated slots existed) and gets handed to a new spawn, which resets its
branch to the pool's clean base and destroys the first task's unlanded work.

state/<id>.meta ownership outlives the agent process, so fm-spawn now refuses
to launch onto a resolved worktree that any other task's metadata still claims,
independent of whether an agent is live in it. The refusal names the owning
task so it can be reconciled or torn down. Physical paths are compared so a
symlinked or relative prefix cannot hide a collision, and the spawn's own task
id is skipped so a recovery re-spawn can reclaim its own recorded slot.

The false-empty pool report itself is a data-loss hazard in the external
treehouse tool, whose availability model has no notion of unlanded git state;
this firstmate-side guard is the durable mitigation regardless of that.

- bin/fm-spawn.sh: worktree_meta_claimant, enforced in validate_spawn_worktree
- tests/fm-spawn-worktree-claim-guard.test.sh: regression (refusal + no false
  positive)
- tests/fm-spawn-dispatch-profile.test.sh: give each distinct task its own
  worktree, as a real pool hands out, so the guard sees separate slots
- bin/fm-test-run.sh: classify the new test in the backend-dispatch family
- docs/architecture.md: document the ownership refusal

Signed-off-by: Lee Calcote <7570704+leecalcote@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Worktree allocation can hand a still-owned pool slot to a new spawn and destroy unlanded work

1 participant