feat: enhance distill prompt with _about.md, folder paths, and daily notes

[cad0p]

Summary

Re-adds the auto-distill-on-shutdown path that was lost when pi-napkin migrated to the upstream simplified model in commit 42970c3 (2026-04-14), and makes concurrent auto-distill safe via per-distill git worktrees.

Closes the gap described in cad0p/pi-napkin#1 for pi 0.68.0+ where session_shutdown now fires reliably with a reason field. Ships alongside PR #10 (/distill-auto-this-session) which was merged earlier.

What changed

Two config-gated features

• distill.onShutdown: boolean (default true) — fires a final detached distill on session_shutdown if the session has new content since the last spawn.
• Per-distill git worktree — every auto-triggered distill (interval OR shutdown) runs in its own git worktree add -b distill/<hash>-<ts>, writes there, then merges back to main via a custom LLM merge driver (napkin-distill-merge). Removes the need for any vault-wide lock.

Setup path (zero user action)

On first session_start with distill.enabled: true where the vault has no .git/:

• Run git init
• Create .gitignore (.napkin/distill/, Obsidian workspace state, common secret patterns) and .gitattributes (*.md merge=napkin-distill-merge) — idempotent merge with any existing content
• Commit initial vault state
• Notify the user once (info severity, includes undo command)

Observability

• /distill-status slash command — lists active distill worktrees (pid, elapsed, branch) and unmerged distill/* branches
• napkin_distill_status pi tool — same data, agent-callable as JSON
• before_agent_start overlap injection — when a running distill is writing to files the current session has also touched, the extension injects a one-time notice into the system prompt (not persisted; zero tokens when no overlap)

Manual /distill stays git-optional

Manual distill path unchanged — no worktree, no git requirement. Only auto-distill (interval + shutdown) switches to the new architecture.

Concurrency model

Each auto-distill:

1. Creates a git worktree off current HEAD on a unique branch
2. Runs pi -p inside the worktree (cwd = worktree root, so napkin resolves vault to the worktree's checkout)
3. Commits the distill's changes
4. git merge <default-branch> with the napkin-distill-merge driver handling any .md conflicts (3-strike LLM retry, pass-by-path with random delimiters for prompt-injection safety, 60s timeout)
5. Partial-merge salvage: files that fail the driver's 3-strike retry revert to main's version; files that merged cleanly keep distill's changes. One error log per discarded file.
6. git merge --squash to main — one clean squash commit per distill
7. git_retry wraps main-mutating commands (no explicit flock — plays with any concurrent git tooling like Obsidian git plugin or user's own autocommit cron)
8. Worktree removed, branch deleted on success

Failed runs leave a line at <vault.configPath>/distill/errors/<ISO-timestamp>-<pid>-<branch-hash>.log (branch name, dangling commit SHA, conflicted files, last LLM response). No worktrees or branches linger.

Tests

pi-napkin had zero tests. This PR adds bun test infrastructure and 230 tests across 14 files:

• shouldDistillOnShutdown predicate — 41 tests (per-guard isolation + integration scenarios + boundary conditions)
• loadVaultConfig — 10 tests
• distill-workspace — worktree lifecycle, meta.json, cleanup
• napkin-distill-merge driver — 9 modes (happy, empty, tiny, huge, no-frontmatter, 3-strike, etc.) + timeout
• git_retry — real index.lock contention
• Wrapper script — end-to-end with real-driver conflict resolution, partial-merge salvage, MERGE_HEAD escape-hatch
• session-touched-files — tool-output shapes + version-pin check against pi's internal extractFileOpsFromMessage
• auto-setup — fresh vault, partial setup, secret patterns, idempotent merge
• Shutdown handler integration — real pi session_shutdown reason values
• before_agent_start handler — overlap, no-overlap, error-path

Docs restructure

README.md is now the single source of truth (~700 lines) covering install, extensions, commands, config, auto-distill setup + concurrency, troubleshooting. skills/napkin/SKILL.md is a thin pointer (~60 lines) preserving the frontmatter pi's skill discovery needs. Eliminates drift between user-facing and agent-facing documentation.

Review process

Three parallel fresh reviewers (correctness, coverage+testing, security+cleanness). 37 findings consolidated, 14 fix-now (1 block + 8 high + 5 medium), rest deferred to v0.1.1 with reasoning. See ~/.napkin/features/pi-napkin-distill/reviews/*.md for the full audit trail. Key issues caught:

• C1: autoDistillSuppressed safety flag was being overwritten by persistence restore — broke setup-failure fail-safe
• C2: meta.json.pid recorded parent pi session pid, not wrapper pid — broke /distill-status liveness
• C3: main was hardcoded in wrapper — silent failure on master-default vaults
• SEC-1: legacy spawnDistill used sh -c with string interpolation — migrated to argv-based spawn
• SEC-3: LLM merge driver was subject to prompt injection via unfenced <<<<<<<< delimiters — switched to pass-by-path with random marker IDs
• G1: no end-to-end test of driver-resolved conflicts surviving the squash path — added

Deferred items (in ~/.napkin/features/pi-napkin-distill/reviews/):

• Intra-file section consolidation (subsumed by future builder-deleter janitor)
• Parallel wrapper test fixture (expensive infra, coverage by unit tests in the meantime)
• Low/nit polish

Known follow-ups (documented for v0.1.1 / future PRs)

• Builder-deleter janitor — [[features/pi-napkin-distill/builder-deleter]] in the vault has the full design (supersedes frontmatter + pressure-triggered consolidation). This PR adds the supersedes: [] convention to the distill prompt as forward-compat, zero behavior change.
• SIGHUP in interactive mode — pi 0.74+ skips session_shutdown on SIGHUP in interactive mode (see earendil-works/pi#4144). Worth an upstream pi issue for an extension-safe path; out of scope here.
• napkin vault subdir-layout migration — cosmetic; a napkin migrate --to-subdir-layout command would move legacy flat vaults to the new .napkin/ subdir layout.

Checklist

• Branch: feat/shutdown-distill-worktree — 39 commits, all SSH-signed
• pnpm install --frozen-lockfile clean (npm → pnpm migration included)
• pnpm lint green
• pnpm test green (230 tests, 0 failures)
• CI updated (GHA workflows use pnpm/action-setup@v4 + pnpm install --frozen-lockfile + bun test)
• README comprehensive, SKILL.md thin pointer (both linked, no drift)
• Auto-distill is opt-in via distill.enabled; manual /distill unchanged
• All 14 triaged review findings addressed with one commit per finding, finding ID in subject
• Design docs in vault: features/pi-napkin-distill/{shutdown-distill,builder-deleter,auto-this-session}.md

Testing

For manual validation:

# In any git-backed napkin vault with distill.enabled: true
# 1. Start a pi session, work for a bit
# 2. /quit
# 3. Observe: napkin vault gets a squash commit ~1-2 min later titled
#    "distill: <YYYY-MM-DD HH:MM>" containing the distilled notes
# 4. /distill-status should show the distill running after /quit

# Concurrency test (two sessions):
tmux new-session -d -s a "pi"
tmux new-session -d -s b "pi"
# Let both fire interval distills; both complete; both commits land on main
git -C "$NAPKIN_VAULT" log --oneline | head
# Expect two "distill: ..." commits, no conflicts in main HEAD