feat: add atomic repository-scoped state creation (#1947)

.changeset/1947-create-if-absent.md

@@ -0,0 +1,13 @@
+---
+"@bradygaster/squad-sdk": minor
+"@bradygaster/squad-cli": minor
+---
+
+Add atomic repository-scoped `createIfAbsent` operation to all state backends (local, git-notes, orphan-branch, two-layer), `StorageProvider` (FSStorageProvider, InMemoryStorageProvider, SQLiteStorageProvider), `StateBackendStorageAdapter`, and `ToolRegistry`/state-mcp MCP surface (`squad_state_create_if_absent`).
+
+- Exactly one concurrent creator succeeds; all others receive `StateKeyConflictError`
+- Existing content is never overwritten
+- Two-layer backend is fail-closed for disagreement/failure between layers
+- New typed errors: `StateKeyConflictError`, `StateBackendUncertaintyError` (exported from `@bradygaster/squad-sdk`)
+- Local backend uses O_CREAT|O_EXCL (filesystem exclusive creation); git-native backends use CAS loops
+- MCP tool `squad_state_create_if_absent` appears in `tools/list` with `conflict`/`uncertainty` failure types

.squad-templates/scribe-charter.md

@@ -77,7 +77,14 @@ perform these checks, **stop and report** rather than proceeding with an unverif
 
 **Worktree awareness:** Use the `TEAM ROOT` provided in the spawn prompt to resolve all `.squad/` paths. If no TEAM ROOT is given, run `git rev-parse --show-toplevel` as fallback. Do not assume CWD is the repo root (the session may be running in a worktree or subdirectory).
 
-**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. Mutable squad state is persisted through runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_delete`, `squad_state_list`, `squad_state_health`) and `squad_decide`. Do not run backend git commands, switch to state branches, push note refs, reset `.squad/`, or commit mutable state by hand. If state tools are unavailable, stop without mutating files or git state and record the tool availability failure in your final summary.
+**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. Mutable squad state is persisted through runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_create_if_absent`, `squad_state_delete`, `squad_state_list`, `squad_state_health`) and `squad_decide`. Do not run backend git commands, switch to state branches, push note refs, reset `.squad/`, or commit mutable state by hand. If state tools are unavailable, stop without mutating files or git state and record the tool availability failure in your final summary.
+
+**Exclusive canonical artifacts:** When exactly one canonical artifact must exist — a retrospective, a session log, a claim marker — create it with `squad_state_create_if_absent`, never `squad_state_write`. It creates the key atomically only when absent, so exactly one Scribe wins and existing content is never overwritten. Handle its two failure shapes explicitly:
+
+- `error: "conflict"` — another Scribe already created the canonical artifact. Do **not** retry as a create and do **not** overwrite. Read the winner's content with `squad_state_read` and append to it if you have something to add.
+- `error: "uncertainty"` — the outcome is unknown. Do **not** assume success and do **not** write over the key. Stop and report the uncertainty in your final summary.
+
+Never emulate this with `squad_state_read` followed by `squad_state_write`; that check-then-write pattern is racy and silently destroys a concurrent Scribe's canonical artifact.
 
 After every substantial work session:
 

.squad-templates/spawn-reference.md

@@ -123,6 +123,11 @@ prompt: |
   whenever they are available:
   - `squad_state_read` / `squad_state_list` for decisions, history, logs, and inbox entries
   - `squad_state_write` / `squad_state_append` for durable updates
+  - `squad_state_create_if_absent` when exactly one canonical artifact must exist
+    (retrospectives, session logs, claim markers). It creates the key atomically
+    only when absent and never overwrites; on `error: "conflict"` another agent
+    already won, and on `error: "uncertainty"` the outcome is unknown — do not
+    assume success and do not overwrite in either case.
   - `squad_state_delete` after Scribe merges inbox entries
   - `squad_state_health` when diagnosing backend availability
   - `squad_decide` for team-relevant decisions

.squad/skills/coordinator-source-of-truth/SKILL.md

@@ -11,7 +11,7 @@ source: "Extracted from squad.agent.md as part of the slimming effort (bradygast
 
 ## State backend note
 
-Files below marked as **"Derived / append-only"** are **mutable state** — agents access them with runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_delete`, `squad_state_list`). The runtime decides whether the configured backend stores them on disk, git-native state, or an external provider. Files marked as **"Authoritative"** are **static config** and always live on disk regardless of backend.
+Files below marked as **"Derived / append-only"** are **mutable state** — agents access them with runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_create_if_absent`, `squad_state_delete`, `squad_state_list`). The runtime decides whether the configured backend stores them on disk, git-native state, or an external provider. Files marked as **"Authoritative"** are **static config** and always live on disk regardless of backend.
 
 ## File hierarchy
 

docs/src/content/docs/features/state-backends.md

@@ -171,12 +171,135 @@ interface StateBackend {
   list(relativeDir: string): string[];
   delete(relativePath: string): boolean;
   append(relativePath: string, content: string): void;
+  createIfAbsent(relativePath: string, content: string): void;
   readonly name: string;
 }
 ```
 
+---
+## Atomic Create-if-Absent
+
+### Overview
+
+`createIfAbsent(key, content)` creates a state key **only when it does not already exist**. Exactly one concurrent caller receives `void` (success); all others receive a typed `StateKeyConflictError`. Existing content is **never overwritten**.
+
+This is the correct primitive for workflows where multiple agents might race to register a session, claim a work item, or initialize shared state — any scenario where exactly one creator must win.
+
+### Repository Scope
+
+The operation is repository-scoped and verified. Before any git-native create,
+the backend asserts that its target directory **is the working-tree root of its
+own repository**, by comparing the realpath of that directory against
+`git rev-parse --show-toplevel`.
+
+This matters because `git rev-parse --git-dir` alone is not enough: Git walks up
+the directory tree, so a non-repository directory nested inside a repository
+silently resolves to the *enclosing* repository. Without the root check, a
+create aimed at a non-repository path would escape its intended scope and write
+state into a repository the caller never named.
+
+Any failure, mismatch, bare repository, or unresolvable path fails closed with
+`StateBackendUncertaintyError`. There is no silent fallback.
+
+### Typed Error Contracts
+
+| Thrown | Meaning | What to do |
+|--------|---------|------------|
+| `StateKeyConflictError` | Key **definitely** already exists. Another creator won. | Read the winner's content; do not retry as a create. |
+| `StateBackendUncertaintyError` | Outcome **unknown** (write failed after exclusive open, CAS retries exhausted, or two-layer disagreement). | Do NOT assume success. Inspect before retrying. |
+
+Neither error is success-shaped. A conflict or uncertainty always surfaces as a thrown error, never as a return value.
+
+### SDK Usage
+
+```typescript
+import {
+  StateKeyConflictError,
+  StateBackendUncertaintyError,
+} from '@bradygaster/squad-sdk';
+
+try {
+  backend.createIfAbsent('sessions/retro-2024-12.md', '# Retro\n');
+  // Success: this process is the sole creator
+} catch (err) {
+  if (err instanceof StateKeyConflictError) {
+    // Another process already created this key — read their content
+    const content = backend.read(err.key);
+  } else if (err instanceof StateBackendUncertaintyError) {
+    // Outcome unknown — do NOT assume success; inspect before retrying
+    console.error('Uncertain create:', err.message);
+  } else {
+    throw err;
+  }
+}
+```
+
+### Per-Backend Guarantees
+
+| Backend | Atomicity mechanism | Conflict detection | Uncertainty condition |
+|---------|--------------------|--------------------|----------------------|
+| **Local** | `open(path, 'wx')` — O_CREAT\|O_EXCL (POSIX + NTFS) | EEXIST on open | Write failure after exclusive open |
+| **Orphan branch** | CAS loop: `update-ref` with expected SHA | Key present at snapshot or at new HEAD after CAS loss | CAS retry limit exhausted with key still absent |
+| **Git notes** | CAS loop: `update-ref` with expected notes-tree SHA | Key present at snapshot or at new HEAD after CAS loss | CAS retry limit exhausted |
+| **Two-layer** | Orphan is authoritative; notes mirrors | Orphan conflict → `StateKeyConflictError` | Orphan success + notes failure/conflict → `StateBackendUncertaintyError` |
+
+**No process-local locks are used.** Atomicity is enforced by OS-level exclusive file creation (local backend) or Git's `update-ref` compare-and-swap (git-native backends).
+
+### Two-Layer Fail-Closed Contract
+
+The two-layer backend uses a **fail-closed** policy for disagreement between layers:
+
+- Orphan layer is authoritative. If it reports conflict → `StateKeyConflictError`.
+- If orphan succeeds but the notes layer fails or reports conflict (disagreement) → `StateBackendUncertaintyError`.
+
+The caller must not assume success on uncertainty. Silent partial success would allow multiple creators to each believe they won.
+
+### MCP Tool Registration
+
+The operation is exposed as the `squad_state_create_if_absent` MCP tool and appears in `tools/list` on state-mcp sessions.
+
+**Tool name:** `squad_state_create_if_absent`  
+**Parameters:** `key` (string, relative to `.squad/`), `content` (string)
+
+Result shapes:
+- `{ resultType: 'success' }` — key created
+- `{ resultType: 'failure', error: 'conflict' }` — key already existed
+- `{ resultType: 'failure', error: 'uncertainty' }` — outcome unknown
+- `{ resultType: 'failure', error: <string> }` — other error (invalid key, etc.)
+
+Only mutable state key prefixes are permitted (`sessions/`, `decisions/inbox/`, `log/`, `orchestration-log/`, `.scratch/`, `identity/`, `agents/*/history.md`, etc.). Static configuration keys (`team.md`, `routing.md`, `config.json`, etc.) are rejected.
+
+### Migration Guidance
+
+If you are currently using `write()` to initialize a key that should only be created once:
+
+```typescript
+// Before (unsafe — unconditionally overwrites):
+if (!backend.exists('sessions/init.md')) {
+  backend.write('sessions/init.md', initialContent);
+}
+
+// After (safe — exactly one concurrent caller succeeds):
+try {
+  backend.createIfAbsent('sessions/init.md', initialContent);
+} catch (err) {
+  if (!(err instanceof StateKeyConflictError)) throw err;
+}
+```
+
+The check-then-write pattern above has a TOCTOU race; `createIfAbsent` closes it.
+
+### Version
+
+`createIfAbsent` and the `squad_state_create_if_absent` MCP tool first ship in
+`@bradygaster/squad-sdk` and `@bradygaster/squad-cli` **0.14.0** (the next minor
+release after 0.13.1). Consumers pinned to `0.12.0` or `0.13.x` must upgrade the
+pin to `0.14.0`, add `squad_state_create_if_absent` to their MCP tool allowlist,
+and restart the state MCP server before the tool appears in `tools/list`.
+
 ---
 ## Security
+
 State backends include hardening against common injection attacks:
 - **Path traversal:** `..` segments are rejected
 - **Null byte injection:** `\0` characters are rejected

docs/src/content/docs/features/storage-provider.md

@@ -31,11 +31,12 @@ All of Squad's data — sessions, decisions, agent memories, event logs — flow
 - **Production** can use SQLite, cloud storage, or a database.
 - **Multi-team deployments** can route different squads to different backends.
 
-The interface is minimal — just 12 core async methods[^1]:
+The interface is minimal — just 13 core async methods[^1]:
 
 ```typescript
 read(filePath: string): Promise<string | undefined>
 write(filePath: string, data: string): Promise<void>
+createIfAbsent(filePath: string, data: string): Promise<void>
 append(filePath: string, data: string): Promise<void>
 exists(filePath: string): Promise<boolean>
 list(dirPath: string): Promise<string[]>
@@ -48,6 +49,12 @@ copy(srcPath: string, destPath: string): Promise<void>
 stat(targetPath: string): Promise<StorageStats | undefined>
 ```
 
+`createIfAbsent` is the only conditional operation: it must create the key
+**atomically and only when absent**, reject with `StateKeyConflictError` when the
+key already exists, and reject with `StateBackendUncertaintyError` when the
+outcome cannot be determined. It must never overwrite existing content.
+See [State Backends → Atomic Create-if-Absent](/features/state-backends/#atomic-create-if-absent).
+
 ---
 
 ## Built-in Providers
@@ -113,6 +120,14 @@ export class MyCustomStorageProvider implements StorageProvider {
     // Create parent directories as needed
   }
 
+  async createIfAbsent(filePath: string, data: string): Promise<void> {
+    // Atomically create ONLY if absent — never overwrite.
+    // Throw StateKeyConflictError if the key already exists.
+    // Throw StateBackendUncertaintyError if the outcome is unknown.
+    // Use a native conditional primitive (O_EXCL open, INSERT OR IGNORE,
+    // If-None-Match: *, conditional PUT) — a read-then-write check is racy.
+  }
+
   async append(filePath: string, data: string): Promise<void> {
     // Append to a file, creating it if missing
   }

packages/squad-cli/src/cli/commands/state-mcp.ts

@@ -31,6 +31,7 @@ const MCP_TOOL_ALIASES: Record<string, string> = {
   squad_state_append: 'squad_state_append',
   squad_state_delete: 'squad_state_delete',
   squad_state_list: 'squad_state_list',
+  squad_state_create_if_absent: 'squad_state_create_if_absent',
   squad_state_health: 'squad_state_health',
   'memory.classify': 'memory.classify',
   'memory.write': 'memory.write',

packages/squad-cli/templates/scribe-charter.md

@@ -77,7 +77,14 @@ perform these checks, **stop and report** rather than proceeding with an unverif
 
 **Worktree awareness:** Use the `TEAM ROOT` provided in the spawn prompt to resolve all `.squad/` paths. If no TEAM ROOT is given, run `git rev-parse --show-toplevel` as fallback. Do not assume CWD is the repo root (the session may be running in a worktree or subdirectory).
 
-**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. Mutable squad state is persisted through runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_delete`, `squad_state_list`, `squad_state_health`) and `squad_decide`. Do not run backend git commands, switch to state branches, push note refs, reset `.squad/`, or commit mutable state by hand. If state tools are unavailable, stop without mutating files or git state and record the tool availability failure in your final summary.
+**State backend awareness:** Check `STATE_BACKEND` from the spawn prompt. Mutable squad state is persisted through runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_create_if_absent`, `squad_state_delete`, `squad_state_list`, `squad_state_health`) and `squad_decide`. Do not run backend git commands, switch to state branches, push note refs, reset `.squad/`, or commit mutable state by hand. If state tools are unavailable, stop without mutating files or git state and record the tool availability failure in your final summary.
+
+**Exclusive canonical artifacts:** When exactly one canonical artifact must exist — a retrospective, a session log, a claim marker — create it with `squad_state_create_if_absent`, never `squad_state_write`. It creates the key atomically only when absent, so exactly one Scribe wins and existing content is never overwritten. Handle its two failure shapes explicitly:
+
+- `error: "conflict"` — another Scribe already created the canonical artifact. Do **not** retry as a create and do **not** overwrite. Read the winner's content with `squad_state_read` and append to it if you have something to add.
+- `error: "uncertainty"` — the outcome is unknown. Do **not** assume success and do **not** write over the key. Stop and report the uncertainty in your final summary.
+
+Never emulate this with `squad_state_read` followed by `squad_state_write`; that check-then-write pattern is racy and silently destroys a concurrent Scribe's canonical artifact.
 
 After every substantial work session:
 

packages/squad-cli/templates/skills/coordinator-source-of-truth/SKILL.md

@@ -11,7 +11,7 @@ source: "Extracted from squad.agent.md as part of the slimming effort (bradygast
 
 ## State backend note
 
-Files below marked as **"Derived / append-only"** are **mutable state** — agents access them with runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_delete`, `squad_state_list`). The runtime decides whether the configured backend stores them on disk, git-native state, or an external provider. Files marked as **"Authoritative"** are **static config** and always live on disk regardless of backend.
+Files below marked as **"Derived / append-only"** are **mutable state** — agents access them with runtime state tools (`squad_state_read`, `squad_state_write`, `squad_state_append`, `squad_state_create_if_absent`, `squad_state_delete`, `squad_state_list`). The runtime decides whether the configured backend stores them on disk, git-native state, or an external provider. Files marked as **"Authoritative"** are **static config** and always live on disk regardless of backend.
 
 ## File hierarchy
 

packages/squad-cli/templates/spawn-reference.md

@@ -123,6 +123,11 @@ prompt: |
   whenever they are available:
   - `squad_state_read` / `squad_state_list` for decisions, history, logs, and inbox entries
   - `squad_state_write` / `squad_state_append` for durable updates
+  - `squad_state_create_if_absent` when exactly one canonical artifact must exist
+    (retrospectives, session logs, claim markers). It creates the key atomically
+    only when absent and never overwrites; on `error: "conflict"` another agent
+    already won, and on `error: "uncertainty"` the outcome is unknown — do not
+    assume success and do not overwrite in either case.
   - `squad_state_delete` after Scribe merges inbox entries
   - `squad_state_health` when diagnosing backend availability
   - `squad_decide` for team-relevant decisions