Skip to content

Add fable-mem: cross-project persistent memory system - #5

Merged
blyatiful1 merged 9 commits into
masterfrom
claude/fable-mem-orchestrate-t7am8v
Jul 9, 2026
Merged

Add fable-mem: cross-project persistent memory system#5
blyatiful1 merged 9 commits into
masterfrom
claude/fable-mem-orchestrate-t7am8v

Conversation

@blyatiful1

Copy link
Copy Markdown
Owner

Introduces a machine-wide, searchable memory corpus that layers on top of Claude's native per-repo memory, closing the gap where decisions banked in one project become invisible in another.

Summary

This adds fable-mem, a new component kind consisting of:

  • A CLI tool (claude/cli/mem.py) for indexing and searching memories across all projects
  • Three protocol hooks for automatic memory management and privacy enforcement
  • Two workflows for memory corpus hygiene and session mining
  • A skill documenting the memory-search capability

The system maintains a disposable sqlite3 FTS5 index over both the global corpus (~/.claude/memory/*.md) and every native per-repo corpus (~/.claude/projects/*/memory/*.md), with graceful degradation to LIKE-based search when FTS5 is unavailable. It uses stdlib only (no pip/venv) so it runs in bare python3 environments.

Key Changes

CLI (claude/cli/mem.py)

  • index: incremental mtime-based indexing with per-file resilience; --rebuild flag for full reconstruction
  • search: keyword recall with scope filtering (global/project), FTS5 with LIKE fallback
  • show: display memory body by ID
  • stats: corpus statistics by scope and project
  • doctor: health check including FTS mode detection and privacy scan
  • gc-scan: mechanical detection of near-duplicates, stale entries, relative dates, and same-topic pairs

Hooks

  • userpromptsubmit-mem-recall.py: injects up to 3 memory pointers (title + description + path, never bodies) as labelled reference data on every prompt; per-session deduplication; fail-open guarantee
  • sessionend-mem-journal.py: appends NDJSON breadcrumb to journal.ndjson (session metadata, git state) and runs incremental reindex; bounded git subprocess budgets to stay under SessionEnd timeout
  • pretool-mem-privacy-guard.py: blocks Write/Edit/MultiEdit into global corpus if pending content matches privacy.toml patterns; fail-open on any ambiguity

Workflows

  • /memory-review: mines session journal for high-activity sessions that banked zero memories; proposes capture candidates via three-way judges
  • /memory-gc: corpus-health sweep combining mechanical gc-scan, contradiction judges for same-topic pairs, relative-date absolutization, and index rebuild; proposals only, never deletes

Configuration

  • privacy.toml: user-owned pattern file (seeded but never overwritten) for work-marker detection
  • Settings snippets wired to PreToolUse (privacy guard) and UserPromptSubmit (recall hook)

Implementation Details

  • BASE resolution: respects CLAUDE_DIR environment variable, falls back to ~/.claude; enables scratch-dir testing without touching $HOME
  • FTS5 probing: availability checked at DB-open with graceful degradation; FABLE_MEM_FORCE_DEGRADED=1 escape hatch for testing
  • WAL mode: enabled for concurrent read/write without blocking
  • Corpus discovery: stable sorted iteration over global + all project memories
  • Frontmatter parsing: minimal one-level YAML with nested metadata: block support; postmortem convention fields (name, description, type, created, verified, visibility)
  • Relative-date detection: regex pattern for hygiene warnings (e.g., "yesterday", "last week")
  • Fail-open posture: read paths (search/show/doctor) catch CorruptIndex gracefully; mutating paths (index/stats/gc-scan) recreate corrupt index; hooks exit 0 on any failure except privacy guard which exits 2 on positive match
  • Deduplication: per-session state file tracks injected memory IDs to avoid repeating pointers within a session

Testing

Comprehensive test suites for each component:

  • test_mem_cli.py: CLI functionality (index, search, show, stats, doctor, gc-scan)
  • test_mem_recall_hook.py: memory injection, deduplication, budget enforcement
  • `test_mem_privacy

https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA

claude added 9 commits July 8, 2026 23:09
…nd search plus example privacy-pattern config

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…and session-journal hooks

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…p and extend postmortem with open-loop/visibility metadata and promotion rules

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…iene workflows (proposals only, never auto-delete/bank)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…tall.sh, settings snippets, and doctor.sh

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…y guard, and recall hook

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…call, non-muting staleness, bounded git budget, WAL, journal privacy sweep, blank-pattern & case-fold guards, doc/CI corrections

Fixes 16 confirmed paranoid-review findings across the fable-mem feature:

mem.py
- Corrupt/unreadable index (DatabaseError) is now handled: mutating commands
  (index/--rebuild, stats, gc-scan) unlink+recreate the disposable index and
  self-heal; search/show fail soft; doctor reports `mode=corrupt` instead of
  tracebacking. Also clears stale WAL sidecars on recreate.
- Non-ASCII query tokenizer: [A-Za-z0-9]+ -> unicode [^\W_]+ to match the fts5
  unicode61 index, so "Café"/"日本語" queries hit instead of silently returning [].
- Blank privacy patterns are filtered (matches the guard), so `doctor --privacy`
  no longer false-positives every file on a stray empty pattern.
- FABLE_MEM_FORCE_DEGRADED is now honored on the readonly search path, and a
  degraded rebuild clears stale fts rows, so the degraded hatch never serves
  stale fts hits.
- `doctor --privacy` sweeps the whole corpus dir (*.md + *.ndjson) so it can see
  the SessionEnd journal, not just markdown.
- WAL journal mode so the read-only recall hook reads a snapshot concurrently
  with an index writer instead of blocking/failing open.

recall hook (userpromptsubmit-mem-recall.py)
- Removed the global staleness mute: a mid-session bank / touched corpus file no
  longer makes recall go dark for the session, and a future-mtimed synced file no
  longer disables recall permanently. Dead pointers (vanished files) are dropped
  per-hit. Query tokenizer unicode-fixed; reader busy_timeout 500->2000ms.

journal hook (sessionend-mem-journal.py)
- git breadcrumbs now bounded by BOTH a 2s per-call timeout AND a 4s total
  wall-clock budget, so slow/hanging git can never delay the breadcrumb append
  past the 10s SessionEnd kill (was up to 3x5s=15s, losing the line).

privacy guard (pretool-mem-privacy-guard.py)
- under_global_corpus case-folds on a case-insensitive fs (probed precisely,
  FABLE_MEM_FS_CASE_INSENSITIVE override) so a capitalized $BASE/Memory/ path
  can't slip a marker past on macOS. Docstring scope caveat added.

CI / docs
- CI compile gate now includes claude/cli/*.py (new component kind).
- README/CHANGELOG/SKILLs: softened the "any write"/"regardless" overclaim to
  "Write|Edit|MultiEdit, not Bash/interpreter writes" + backstop note; dropped the
  phantom tests/test_memory_skill.py; corrected suite count 110->148 to 111->163.

Every fix has extended test coverage; full suite green (163 passed), workflows ok.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
@blyatiful1
blyatiful1 merged commit be09e92 into master Jul 9, 2026
2 checks passed
@blyatiful1
blyatiful1 deleted the claude/fable-mem-orchestrate-t7am8v branch July 9, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants