Add persistent-memory primitives: live_extract, contradiction_pipeline, correction_gate

[agulaya24]

What this adds

Three package modules under src/baselayer/ that form the stable per-conversation ingestion/consistency layer for the planned conversation-capture proxy (D-049):

• live_extract.py: single-conversation extraction orchestrator. Wraps extract_facts.process_conversation, snapshots fact IDs before extraction, reports each newly stored fact as an ADD action, runs the contradiction hook on new state-facts, and returns a change manifest (conversation_id, facts_stored, actions, contradictions_found, probes_queued, optional error). Also supports import-then-extract for a single file. Runnable as python -m baselayer.live_extract.
• contradiction_pipeline.py: per-fact consistency check. ChromaDB similarity search over active state-facts, LLM classification into contradiction / enrichment / coexistent / ambiguous, then execution: contradictions supersede the older fact, ambiguous pairs land in a probe_queue table, enrichment and coexistence take no action. Follows D-036 (Binary Collapse Resistance: default coexistent on any failure) and skips events (immutable).
• correction_gate.py: query-time fact filtering for the serving layer. Pass 1 drops superseded_by IS NOT NULL; pass 2 drops facts with a delete/supersede row in user_corrections. Corrections take effect immediately without re-running the pipeline.

Placement reasoning

These were loose scripts under scripts/, which a recent commit is actively cleaning out (the orphaned mirror was retired). They are library code with a small public API that the capture proxy will import per conversation, so they belong in the package: absolute baselayer.* imports, no sys.path mutation, shipped by the existing packages = ["baselayer"] setuptools config, testable under the standard suite.

Scrubbed / hardened

• No personal names, machine paths, or keys were present in the originals; verified again after the move (no C:\Users, no subject names beyond the public examples, no key material).
• All constants and paths routed through config.py. Classification model now routed through LLM_PROVIDER_CONFIG["contradiction"] (env-overridable via BASELAYER_LLM_CONTRADICTION) instead of a hardcoded model constant.
• correction_gate rewired from a nonexistent fact_corrections table to the standard user_corrections schema (original_fact_id, correction_type matched case-insensitively; flag corrections do not remove facts). IN(...) lists chunked below SQLite's host-parameter limit.
• probe_queue.created_at set explicitly at insert time; the previous unixepoch() column default requires SQLite 3.38+.
• LLM judgments outside the four valid categories are rejected and degrade to coexistent.
• Extraction and contradiction-hook failures are logged and reported in the manifest instead of raising; silent bare excepts replaced with logging on the paths that matter.
• Dead code removed (unused imports, an unused source-conversation lookup in cascade detection).

Test coverage

29 new tests in tests/test_memory_primitives.py, using the existing temp_db/populated_db fixtures, all API calls mocked:

• live_extract: manifest shape with new-fact ADD actions, pre-existing facts not re-reported, NOOP path, conversation-not-found, extraction failure, contradiction hook invocation and hook-failure isolation.
• contradiction_pipeline: classification parsing (clean JSON, embedded JSON, invalid judgment, API error), judgment execution per verdict (supersession, probe queueing, no-op), batch dispatch counting, event/incomplete-fact skipping, similarity-search filtering (self, threshold, superseded).
• correction_gate: superseded filtering, delete/supersede corrections (both cases), flag corrections retained, missing-table tolerance, large-list chunking, get_active_facts shape and subject filter.

Full suite: 480 passed (451 existing + 29 new). Doc test counts updated (CLAUDE.md and AGENTS.md were already stale at 403; CONTRIBUTING at 414).

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fb8f1776-036c-4d85-9e3c-a2ea375f5473

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch memory-primitives

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}