fix(spawn): refuse a worktree slot another task's metadata still claims - #1443
Open
leecalcote wants to merge 1 commit into
Open
fix(spawn): refuse a worktree slot another task's metadata still claims#1443leecalcote wants to merge 1 commit into
leecalcote wants to merge 1 commit into
Conversation
The worktree pool decides slot reuse from live-agent presence, not recorded task ownership. A crash- or reboot-orphaned but still-owned slot reads as "empty" (observed 2026-07-21, when treehouse reported the pool empty while six populated slots existed) and gets handed to a new spawn, which resets its branch to the pool's clean base and destroys the first task's unlanded work. state/<id>.meta ownership outlives the agent process, so fm-spawn now refuses to launch onto a resolved worktree that any other task's metadata still claims, independent of whether an agent is live in it. The refusal names the owning task so it can be reconciled or torn down. Physical paths are compared so a symlinked or relative prefix cannot hide a collision, and the spawn's own task id is skipped so a recovery re-spawn can reclaim its own recorded slot. The false-empty pool report itself is a data-loss hazard in the external treehouse tool, whose availability model has no notion of unlanded git state; this firstmate-side guard is the durable mitigation regardless of that. - bin/fm-spawn.sh: worktree_meta_claimant, enforced in validate_spawn_worktree - tests/fm-spawn-worktree-claim-guard.test.sh: regression (refusal + no false positive) - tests/fm-spawn-dispatch-profile.test.sh: give each distinct task its own worktree, as a real pool hands out, so the guard sees separate slots - bin/fm-test-run.sh: classify the new test in the backend-dispatch family - docs/architecture.md: document the ownership refusal Signed-off-by: Lee Calcote <7570704+leecalcote@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
fm-spawn.shcan be handed a worktree pool slot that another in-flight task still owns, launch a new agent on top of it, and silently destroy the first task's unlanded work. Observed 2026-07-21:treehouse statusreported the pool as effectively empty while six populated, mid-flight slots existed, and a new spawn reset an in-flight task's worktree to a detached HEAD at the pool's clean base.Root cause
Worktree availability is decided from live-agent presence, not recorded task ownership. After
treehouse get(ororca worktree create) resolves a worktree,validate_spawn_worktreeonly checks that the path is a real git worktree distinct from the primary checkout - it never checks whether anystate/<id>.metastill records that path as an owned task's worktree. So once the pool hands back a slot whose owner process has died (host reboot, thetreehouse getsubshell exiting while the agent persists, PID/start-time drift), firstmate launches on it unconditionally.state/<id>.metaownership outlives the agent process, so firstmate holds durable ownership the pool does not.Fix
bin/fm-spawn.sh: addworktree_meta_claimantand enforce it invalidate_spawn_worktree. The spawn now refuses to launch onto a resolved worktree that any other task'sstate/<id>.metastill claims, independent of whether an agent is live in it. The refusal names the owning task so it can be reconciled or torn down. Physical (realpath) paths are compared so a symlinked or relative prefix cannot hide a real collision, and the spawn's own task id is skipped so a recovery re-spawn can reclaim its own recorded slot. Living invalidate_spawn_worktreecovers treehouse and Orca (and any future backend) at the single chokepoint.Note on the pool-side cause
The false-empty pool report itself is a data-loss hazard in the external
kunchenguid/treehousetool, whose availability model has no notion of unlanded git state. This change is the durable firstmate-side guard that fails closed regardless; a companion fix is being filed againstkunchenguid/treehouseso thatget, when reclaiming a slot whoseowner_pidis dead, refuses to destructively reset a worktree that is dirty or ahead of its base.Verification
tests/fm-spawn-worktree-claim-guard.test.sh(new): drives the realfm-spawn.shand asserts a meta-claimed slot is refused (exit 1, names the owner, leaves the claimant's meta intact, writes no new meta) and that a claimant owning a different worktree does not false-refuse a free slot.tests/fm-spawn-dispatch-profile.test.sh: updated so distinct tasks get distinct worktrees (as a real pool hands out), since two tasks sharing one fake slot now correctly trips the guard; includes a small opt-in per-window worktree map in its fake tmux for the batch case.bin/fm-test-run.sh: the new test is classified in thebackend-dispatchfamily;--check-coveragebalances.docs/architecture.md: documents the ownership refusal.bin/fm-lint.shpasses; the changed-surface tests pass.Closes #1441
Updates from git push no-mistakes