Add fable-mem: cross-project persistent memory system

.github/workflows/ci.yml

@@ -13,7 +13,7 @@ jobs:
         run: bash -n install.sh tools/doctor.sh bench/run.sh bench/task/verify.sh
 
       - name: Python components compile
-        run: python3 -m py_compile claude/hooks/*.py bench/score.py bench/task/loglib/*.py
+        run: python3 -m py_compile claude/hooks/*.py claude/cli/*.py bench/score.py bench/task/loglib/*.py
 
       - name: Settings snippet is valid JSON
         run: python3 -c "import json; json.load(open('claude/settings/settings-snippet.json'))"

CHANGELOG.md

@@ -1,5 +1,80 @@
 # Changelog
 
+## v1.8 — 2026-07-08
+
+Memory pass: the kit stops forgetting across projects. Native auto-memory is per-git-repo,
+so a decision banked in one repo is invisible in the next — fable-mem layers a machine-wide,
+searchable memory corpus **on top of** the native one (never wrapping it) at the unclaimed
+`~/.claude/memory/`, with the kit's usual posture: deterministic where it must hold, quiet
+where it would annoy, fail-open everywhere. Embeddings stay a deliberate non-goal for v1
+(stdlib sqlite3 FTS5 only; the vector index waits for ~500 memories or demonstrated
+synonym-recall misses).
+
+### Added
+- **`cli/mem.py` — a new component kind.** A single stdlib-only Python file (sqlite3 FTS5,
+  no pip/venv) exposing `index · search · show · stats · doctor · gc-scan` over the L1 global
+  corpus (`~/.claude/memory/*.md`) AND every native per-repo corpus
+  (`~/.claude/projects/*/memory/*.md`), each row scope-tagged. FTS5 is probed at DB-open and
+  degrades to a plain-table `LIKE` scan when absent; `doctor` reports the active mode. The
+  index is disposable (rebuildable from the corpus), commits per-file so a timeout-kill
+  preserves progress, and never mutates a memory file. Because it isn't a hook/agent/
+  workflow/skill, it gets hand-written `install.sh` copy + bootstrap and `tools/doctor.sh`
+  check blocks (the four existing globs don't see it).
+- **Cross-project recall hook (`userpromptsubmit-mem-recall.py`, UserPromptSubmit).** One
+  read-only FTS query per prompt injects at most three memory pointers (title + one-line
+  description + path — never bodies) as inert, labelled reference data. Threshold-gated,
+  ~600-token budget, per-session dedupe under `FABLE_STATE_DIR`; opens the index `mode=ro`
+  and never builds on the prompt path (stale → silent). Fail-open: a malformed payload, a
+  missing/locked index, or any bug ends in `exit 0` with no output, so the prompt is never
+  lost. Prompt-injection-inert formatting (control chars stripped, quoted refs, never file
+  bodies).
+- **Session-journal hook (`sessionend-mem-journal.py`, SessionEnd).** Appends exactly one
+  NDJSON breadcrumb per session (ISO ts, cwd, git root + branch + dirty-file count, end
+  reason — computed via `git` subprocesses bounded by BOTH a per-call `timeout=` AND a small
+  total wall-clock budget, since SessionEnd carries no native metadata beyond `reason`) to
+  `~/.claude/memory/journal.ndjson`, rotates at 5MB, then runs an incremental `mem index` so
+  this session's memory is searchable next session. The line is written **before** the reindex,
+  and both settings snippets declare an explicit `"timeout": 10` (the SessionEnd default is
+  1.5s, which would kill the hook and lose the breadcrumb); the total git budget stays well
+  under that 10s so a slow/hanging git can never delay the append past the kill. Fail-open.
+- **Privacy-guard hook (`pretool-mem-privacy-guard.py`, PreToolUse on `Write|Edit|MultiEdit`).**
+  The deterministic project→global promotion gate: a `Write|Edit|MultiEdit` whose target
+  resolves under `~/.claude/memory/` has its pending content (`content`/`new_string`) scanned
+  against the user's `privacy.toml` work-markers; a hit `exit 2`s and blocks the write
+  **before** the marker lands. It matches those tools, not Bash/interpreter writes (`cat >>`,
+  `python3 -c`) — `mem doctor --privacy` is the backstop for those. Writes outside the corpus
+  and clean payloads pass untouched; unloadable patterns fail open (a guard that can't read
+  patterns can't honestly block). Advisory SKILL.md text was never enough — under momentum the
+  model promotes anyway.
+- **`memory-search` skill** — search the machine-wide corpus before re-deriving a decision
+  already made in another repo; when to search, when NOT (facts visible in the current
+  repo/git/CLAUDE.md), and how to promote a project lesson to global.
+- **`/memory-review` workflow** (`claude/workflows/memory-review.js`) — mines the session
+  journal for high-activity sessions that banked ZERO cross-project memories, then a
+  three-way judge PROPOSES the durable lessons worth capturing. On-demand only; proposes,
+  never writes (banking still happens explicitly via postmortem).
+- **`/memory-gc` workflow** (`claude/workflows/memory-gc.js`) — corpus-health sweep:
+  mechanical `mem gc-scan` (near-dup / stale / relative-date / same-topic candidates) →
+  LLM contradiction judges with three-way verdicts on same-topic pairs → absolutize relative
+  dates in place → rebuild the disposable index → report. NEVER deletes — every removal comes
+  back as a proposal. (The mechanical `gc-scan` layer is test-covered; the workflow's LLM
+  verdict layer is compile-checked only, like every workflow, and labelled unproven-by-harness.)
+- **`postmortem` skill → v2** — a promotion rule (project→global only when explicit, with a
+  one-line why-global; advisory, since the deterministic gate is the privacy-guard hook), a
+  `visibility: private|shareable` field, an `open-loop` memory type, and hygiene rules
+  (falsifiable conclusions, update-don't-duplicate, delete-what-evidence-refutes).
+- **`privacy.toml.example`** — conservative, ships EMPTY (necessary-not-sufficient by design);
+  `install.sh` copies it to `~/.claude/memory/privacy.toml` only if absent and bootstraps
+  `mem index --rebuild` once so the corpus is indexed before any SessionEnd fires.
+- Doctrine pointer (`claude/CLAUDE.md`): search the cross-project corpus before re-deriving,
+  promote worth-keeping lessons global via postmortem, run `/memory-gc` when the corpus feels
+  stale. README gains a fable-mem section, tree entries, playbook rows, and three Known-limits
+  entries (the SessionEnd crash gap, patterns-necessary-not-sufficient, per-subagent memory
+  islands).
+- `tests/test_mem_cli.py`, `tests/test_mem_recall_hook.py`, `tests/test_mem_journal_hook.py`,
+  `tests/test_mem_privacy_guard.py`, plus the settings-snippet expected-dict (three new
+  hook→event rows) and the stateful-hook consistency tuple (recall hook). Suite: 111 → 163.
+
 ## v1.7 — 2026-07-07
 
 Design pass: the kit learns to design websites, not just verify code — with the

README.md

@@ -49,6 +49,12 @@ claude/
                                self-contained HTML previews → distinct-lens judges
                                (+ German-compliance judge when the brief says German)
                                → synthesis: winner + what to graft from the losers
+    memory-review.js           /memory-review — mine the session journal for
+                               high-activity sessions that banked nothing → propose
+                               capture candidates (on-demand; proposes, never writes)
+    memory-gc.js               /memory-gc — corpus-health sweep: mechanical gc-scan +
+                               three-way contradiction judges + date-absolutize +
+                               index rebuild; proposals only, never deletes
   skills/
     fable/                     the flagship: full staged protocol for hard tasks (/fable)
     webdesign/                 web design protocol: explicit design views (static /
@@ -59,7 +65,10 @@ claude/
                                (live-researched, sources cited, claims adversarially
                                verified at authoring time)
     orchestrate/               multi-agent workflow authoring playbook
-    postmortem/                distill lessons into persistent memory
+    postmortem/                distill lessons into persistent memory (promote the
+                               cross-project ones global — explicit + one-line why)
+    memory-search/             search the machine-wide cross-project corpus before
+                               re-deriving a decision already made in another repo
   hooks/
     stop-claim-audit.py        blocks the first "done/verified" stop after file edits
                                (Edit/Write or file-writing Bash), forces one audit pass
@@ -81,7 +90,23 @@ claude/
                                compaction (per-session state file)
     sessionstart-compact-recovery.py  post-compaction injection: recovery protocol +
                                the saved original request + the ACTUAL git state
-  settings/settings-snippet.json   effortLevel xhigh + all six hooks wired to their events
+    userpromptsubmit-mem-recall.py  cross-project memory recall: one read-only FTS
+                               query per prompt injects ≤3 memory pointers (title +
+                               description + path, never bodies) as inert refs —
+                               threshold-gated, per-session dedupe, fail-open
+    sessionend-mem-journal.py  appends one NDJSON breadcrumb per session +
+                               incremental reindex, so this session's memory is
+                               searchable in the next one; git-bounded, fail-open
+    pretool-mem-privacy-guard.py  blocks a Write/Edit into the global corpus whose
+                               pending content hits a privacy.toml work-marker — the
+                               deterministic project→global promotion gate
+  cli/                         standalone CLI (new component kind, stdlib-only)
+    mem.py                     the fable-mem index/recall CLI (sqlite3 FTS5, no pip):
+                               index · search · show · stats · doctor · gc-scan over
+                               the global + every per-repo corpus
+    privacy.toml.example       work-marker patterns; installed to
+                               ~/.claude/memory/privacy.toml only if absent
+  settings/settings-snippet.json   effortLevel xhigh + all nine hooks wired to their events
   settings/settings-snippet-small.json  same, plus FABLE_LOOP_THRESHOLD=2 for small drivers
 install.sh                     copies into ~/.claude with out-of-tree backups; idempotent;
                                never edits settings. Small-driver flags: --tier small
@@ -107,6 +132,47 @@ with zero false claims**, and in one run the transcript shows the gate directly
 would-be false claim — the model tried to stop, got blocked, ran the check it had skipped,
 and fixed the bug it had shipped. Small n, honest stats in the file.
 
+## Cross-project memory (fable-mem)
+
+Claude Code's native auto-memory is per-git-repo: a decision banked in repo A is invisible
+while you work in repo B, so the same wheel gets reinvented across projects. fable-mem layers
+a machine-wide memory corpus **on top of** the native one — never wrapping it, only adding a
+shared, searchable cross-project surface at `~/.claude/memory/` (unclaimed by any native
+feature). It carries the same discipline as the rest of the kit: deterministic where it must
+hold, quiet where it would annoy, fail-open everywhere.
+
+- **Recall without asking.** A UserPromptSubmit hook runs one read-only FTS5 query against a
+  local sqlite index and injects at most three memory pointers (title + one-line description
+  + path — never bodies) as inert, labelled reference data. Threshold-gated, ~600-token
+  budget, per-session dedupe: silence over noise. Cross-repo, so a lesson from project A
+  surfaces while you work in project B.
+- **A breadcrumb every session.** A SessionEnd hook appends one NDJSON line (timestamp, cwd,
+  git root + branch + dirty-file count, end reason) to `~/.claude/memory/journal.ndjson` — a
+  deterministic trace even when the session banked nothing — then runs an incremental reindex
+  so this session's memory is searchable in the next. `/memory-review` mines that journal for
+  high-activity sessions that banked nothing and proposes what was worth keeping.
+- **The promotion boundary is a hook, not a rule.** The one line that must hold is project →
+  global: a work marker (internal ticket id, private hostname, client codename) must never
+  cross into the shared corpus. A PreToolUse guard scans the pending content of any
+  **Write/Edit/MultiEdit** into `~/.claude/memory/` against your `privacy.toml` and blocks it
+  (exit 2) before the marker lands; it matches those tools, not Bash/interpreter writes
+  (`cp`/`cat >>`/`python3 -c`), so `mem doctor --privacy` is the detective backstop that sweeps
+  the whole corpus dir — including the `.ndjson` journal — for anything the write-time gate
+  didn't see.
+- **Hygiene that proposes, never deletes.** `mem gc-scan` mechanically flags near-duplicates,
+  stale entries, relative-date offenders, and same-topic pairs; `/memory-gc` adds three-way
+  contradiction judges and rebuilds the index. Every removal comes back as a proposal — the
+  corpus is never mutated out from under you.
+- **Verifiable install.** `./tools/doctor.sh` checks the CLI compiles and reports its FTS
+  mode, that the memory dir is writable, and that all three hooks are wired — the same
+  no-silently-inert guarantee the rest of the kit gets.
+
+The index is stdlib-only (sqlite3 FTS5, no pip/venv, no daemon or cron) and disposable —
+rebuilt from the corpus at any time. **Embeddings are a deliberate non-goal for v1**: reach
+for a vector index only when the corpus exceeds ~500 memories, or when keyword recall
+demonstrably misses on synonym-heavy queries (the right memory exists but shares no surface
+tokens with the prompt). Until then, FTS5 keyword recall carries it.
+
 ## Install
 
 ```bash
@@ -140,6 +206,8 @@ Finally, confirm the doctrine load in a fresh session: *"quote the first bullet
 | Bug survives two fix attempts | `oracle` agent |
 | Work one context can't hold | `orchestrate` skill |
 | End of a debugging saga | `postmortem` skill |
+| About to re-derive a decision you may have made in another repo | `memory-search` skill — search the cross-project corpus first |
+| Cross-project memory corpus feels stale | `/memory-gc` — propose dedupes / contradictions / stale, never delete |
 
 ## Running under ultracode
 
@@ -199,6 +267,9 @@ Going the other direction — running the kit on a **smaller** driver model (a S
 - The loop-alarm hook counts a run as *failed* only when the PostToolUse payload carries an explicit exit code / error flag. If your Claude Code version omits exit information from Bash `tool_response`, the alarm is silently inert (fail-open by design) — verify once with a deliberately failing command repeated 3×.
 - The test-weakening alarm reads Edit/Write payloads, so a skip marker smuggled in via a Bash heredoc doesn't trip it at edit time — but the claim-audit gate now flags any file-writing Bash command that names a test path, so the stop-time audit still fires.
 - The destructive-command guard is a tripwire, not a jail: known bypass classes include commands wrapped in `sh -c '...'`, destructive flags hidden by quoting (`git reset '--hard'`), and some `rm -rf` variants (`~/*`, `./*`). The claim-audit gate similarly misses file writes done through interpreters (`python3 -c`) and some multi-line Bash forms. These hooks raise the cost of the documented *reflexive* failure modes; they do not stop a determined evader — pair them with the doctrine, and treat any deliberate bypass in a transcript as the incident.
+- The fable-mem session journal and its reindex run on SessionEnd, which fires on graceful exit (`/clear`, resume, logout, quit) but is **not** guaranteed on a hard crash or SIGKILL — a session killed mid-flight leaves no breadcrumb, and its memory waits for the next SessionEnd to be indexed. The corpus files are never at risk (the model writes them during the session); only the journal line and index freshness are.
+- The privacy guard's `privacy.toml` patterns are **necessary, not sufficient**: they block the markers you list, not the ones you forgot. The list ships empty and conservative so a fresh install never false-positives — which means it catches nothing until you fill in your real work markers. Treat it as a tripwire for known-shaped leaks, not a classifier, and run `mem doctor --privacy` before promoting. The guard is also **tool-scoped**: it fires on `Write|Edit|MultiEdit` into the corpus, not on Bash/interpreter writes (`cp`/`mv`/`cat >>`/`python3 -c`) — the same interpreter-bypass class the destructive-guard and claim-audit gates document — so a promotion done by copying rather than re-writing lands unscanned; `mem doctor --privacy` (which now sweeps the `.ndjson` journal too, not just `*.md`) is the backstop.
+- fable-mem claims `~/.claude/memory/` because no native feature uses it: main-session auto-memory is per-repo (`~/.claude/projects/<p>/memory/`) and native "user scope" memory is **per-subagent islands** (`~/.claude/agent-memory/<name>/`), not a shared cross-project store. If a future Claude Code ships a real shared user-memory surface at that path, re-check for collision before upgrading.
 - No prompt kit closes the gap on the longest-horizon work (multi-hour autonomous runs); route those to a stronger model when available.
 - Built for Claude Code 2.1.x in mid-2026; contracts (workflow API, hook events, frontmatter) may drift. The v1.1 components were verified live on `claude-opus-4-8` + Claude Code 2.1.198 on 2026-07-02; components added since (v1.2+ hooks, doctor, small-tier profile, /big-task) are covered by the unit suite and workflow checker but have not all had a live session pass — run `./tools/doctor.sh` and the one-minute live checks after installing.
 

claude/CLAUDE.md

@@ -17,6 +17,7 @@ Succession package written by Claude Fable 5 (2026-07-02) to run Claude Opus 4.8
 ## Reach for tools early — you under-trigger by default
 - Version-sensitive, fast-moving, or post-cutoff library/API questions: check live docs or WebSearch instead of trusting training memory. Stable stdlib basics need no lookup.
 - You under-use persistent memory too: before re-deriving a decision about this project, check auto-memory (MEMORY.md); when a saga ends with a non-obvious lesson, bank it (postmortem skill) instead of letting it die with the session.
+- Cross-project memory (fable-mem): native MEMORY.md only covers THIS repo, so before re-deriving a decision you may have made elsewhere, search the machine-wide corpus (`python3 ~/.claude/cli/mem.py search "<terms>"`, or the memory-search skill). Promote a lesson worth other projects to the global corpus via postmortem (the privacy guard blocks work-markers from leaking); run `/memory-gc` when the corpus feels stale.
 - Broad code searches: delegate to Explore subagents instead of grepping serially in your own context.
 - A bug survives two fix attempts: stop grinding, hand ALL evidence to the `oracle` agent. If the oracle's next experiment also dead-ends, the ladder ends at the human: hand them a decision-ready summary (dead hypotheses one line each, surviving candidates, the experiment you'd run next) — never a third lap of the same loop.
 - Before multi-file or unfamiliar work: plan first (use /deep-plan when the strategy is genuinely open-ended), then have the `plan-critic` agent attack the plan before you write code.