Add fable-mem: cross-project persistent memory system - #5
Merged
Conversation
…nd search plus example privacy-pattern config Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…and session-journal hooks Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…p and extend postmortem with open-loop/visibility metadata and promotion rules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…iene workflows (proposals only, never auto-delete/bank) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…tall.sh, settings snippets, and doctor.sh Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…y guard, and recall hook Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…tion Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
…call, non-muting staleness, bounded git budget, WAL, journal privacy sweep, blank-pattern & case-fold guards, doc/CI corrections Fixes 16 confirmed paranoid-review findings across the fable-mem feature: mem.py - Corrupt/unreadable index (DatabaseError) is now handled: mutating commands (index/--rebuild, stats, gc-scan) unlink+recreate the disposable index and self-heal; search/show fail soft; doctor reports `mode=corrupt` instead of tracebacking. Also clears stale WAL sidecars on recreate. - Non-ASCII query tokenizer: [A-Za-z0-9]+ -> unicode [^\W_]+ to match the fts5 unicode61 index, so "Café"/"日本語" queries hit instead of silently returning []. - Blank privacy patterns are filtered (matches the guard), so `doctor --privacy` no longer false-positives every file on a stray empty pattern. - FABLE_MEM_FORCE_DEGRADED is now honored on the readonly search path, and a degraded rebuild clears stale fts rows, so the degraded hatch never serves stale fts hits. - `doctor --privacy` sweeps the whole corpus dir (*.md + *.ndjson) so it can see the SessionEnd journal, not just markdown. - WAL journal mode so the read-only recall hook reads a snapshot concurrently with an index writer instead of blocking/failing open. recall hook (userpromptsubmit-mem-recall.py) - Removed the global staleness mute: a mid-session bank / touched corpus file no longer makes recall go dark for the session, and a future-mtimed synced file no longer disables recall permanently. Dead pointers (vanished files) are dropped per-hit. Query tokenizer unicode-fixed; reader busy_timeout 500->2000ms. journal hook (sessionend-mem-journal.py) - git breadcrumbs now bounded by BOTH a 2s per-call timeout AND a 4s total wall-clock budget, so slow/hanging git can never delay the breadcrumb append past the 10s SessionEnd kill (was up to 3x5s=15s, losing the line). privacy guard (pretool-mem-privacy-guard.py) - under_global_corpus case-folds on a case-insensitive fs (probed precisely, FABLE_MEM_FS_CASE_INSENSITIVE override) so a capitalized $BASE/Memory/ path can't slip a marker past on macOS. Docstring scope caveat added. CI / docs - CI compile gate now includes claude/cli/*.py (new component kind). - README/CHANGELOG/SKILLs: softened the "any write"/"regardless" overclaim to "Write|Edit|MultiEdit, not Bash/interpreter writes" + backstop note; dropped the phantom tests/test_memory_skill.py; corrected suite count 110->148 to 111->163. Every fix has extended test coverage; full suite green (163 passed), workflows ok. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces a machine-wide, searchable memory corpus that layers on top of Claude's native per-repo memory, closing the gap where decisions banked in one project become invisible in another.
Summary
This adds fable-mem, a new component kind consisting of:
claude/cli/mem.py) for indexing and searching memories across all projectsThe system maintains a disposable sqlite3 FTS5 index over both the global corpus (
~/.claude/memory/*.md) and every native per-repo corpus (~/.claude/projects/*/memory/*.md), with graceful degradation to LIKE-based search when FTS5 is unavailable. It uses stdlib only (no pip/venv) so it runs in barepython3environments.Key Changes
CLI (
claude/cli/mem.py)index: incremental mtime-based indexing with per-file resilience;--rebuildflag for full reconstructionsearch: keyword recall with scope filtering (global/project), FTS5 with LIKE fallbackshow: display memory body by IDstats: corpus statistics by scope and projectdoctor: health check including FTS mode detection and privacy scangc-scan: mechanical detection of near-duplicates, stale entries, relative dates, and same-topic pairsHooks
userpromptsubmit-mem-recall.py: injects up to 3 memory pointers (title + description + path, never bodies) as labelled reference data on every prompt; per-session deduplication; fail-open guaranteesessionend-mem-journal.py: appends NDJSON breadcrumb tojournal.ndjson(session metadata, git state) and runs incremental reindex; bounded git subprocess budgets to stay under SessionEnd timeoutpretool-mem-privacy-guard.py: blocks Write/Edit/MultiEdit into global corpus if pending content matchesprivacy.tomlpatterns; fail-open on any ambiguityWorkflows
/memory-review: mines session journal for high-activity sessions that banked zero memories; proposes capture candidates via three-way judges/memory-gc: corpus-health sweep combining mechanical gc-scan, contradiction judges for same-topic pairs, relative-date absolutization, and index rebuild; proposals only, never deletesConfiguration
privacy.toml: user-owned pattern file (seeded but never overwritten) for work-marker detectionImplementation Details
CLAUDE_DIRenvironment variable, falls back to~/.claude; enables scratch-dir testing without touching$HOMEFABLE_MEM_FORCE_DEGRADED=1escape hatch for testingmetadata:block support; postmortem convention fields (name, description, type, created, verified, visibility)CorruptIndexgracefully; mutating paths (index/stats/gc-scan) recreate corrupt index; hooks exit 0 on any failure except privacy guard which exits 2 on positive matchTesting
Comprehensive test suites for each component:
test_mem_cli.py: CLI functionality (index, search, show, stats, doctor, gc-scan)test_mem_recall_hook.py: memory injection, deduplication, budget enforcementhttps://claude.ai/code/session_01Vko5e15eEpKpFNZBJBYoDA