Skip to content

feat(persona): engram recall surface (#1121 PR-5)#1163

Merged
joelteply merged 1 commit into
canaryfrom
feat/engram-recall-surface
May 14, 2026
Merged

feat(persona): engram recall surface (#1121 PR-5)#1163
joelteply merged 1 commit into
canaryfrom
feat/engram-recall-surface

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

Summary

Closes the read-side of the engram thread. Admission state from PR-4 already accumulates engrams per-persona; this PR adds the typed query API + IPC handler so callers can actually retrieve them.

Card

continuum#1162.

What ships

  • Recall queries on AdmissionState:
    • recall_recent(limit) — newest-first N engrams
    • recall_by_id(id) — exact lookup
    • recall_by_keyword(keyword, limit) — case-insensitive substring, newest-first, limit-capped
    • recall_by_origin_kind(kind, limit) — filter by Chat / Airc / Tool / SelfReflection
  • EngramOriginKind discriminator enum + From<&EngramOrigin> impl — exhaustive match means new origin variants force compile-time update (so the recall filter never silently misses a new variant).
  • cognition/recall-engrams IPC handler — kind=recent|by_id|by_keyword|by_origin + standard params. Returns { engrams, count } JSON. Defaults to kind=recent + limit=10.

Scope (sliced)

  • ⏭️ PR-6: ORM persistence (engrams still in-memory; queries hit the Vec). Backing store swaps without changing this API.
  • ⏭️ PR-7+: Embedding-based / semantic recall (keyword is substring only for v1).
  • ⏭️ Pagination cursors (limit is the only knob; assumption: callers want the most recent slice).

Validation

$ cargo test -p continuum-core --features metal,accelerate persona::admission_state
test result: ok. 15 passed; 0 failed; 0 ignored

6 new tests:

  • recall_recent_returns_newest_first (newest-first ordering invariant)
  • recall_recent_respects_limit_above_and_below_count (limit edge cases)
  • recall_by_id_finds_known_returns_none_unknown (exact lookup)
  • recall_by_keyword_case_insensitive_newest_first_with_limit (search semantics + empty-needle skip)
  • recall_by_origin_kind_filters_to_requested_variant (filter correctness)
  • engram_origin_kind_covers_all_origin_variants (compile-time exhaustive From)

cargo build --lib clean. npm run build:ts clean. Hooks ran without --no-verify.

Closes engram thread substrate

PRs 1-5 + fix #1157 all merged on canary. Next slice options:

  • PR-6: ORM persistence (admitted engrams survive restart, queryable via data/list)
  • TS-side wiring (separate slice): call cognition/admit-inbox-message + cognition/recall-engrams from the chat path so personas actually accumulate + reference their memory in real conversations

🤖 Generated with Claude Code

Closes the read-side of the engram thread. Admission state from PR-4
already accumulates engrams per-persona; this PR adds the typed query
API + IPC handler so callers can actually retrieve them.

What ships:

- `AdmissionState::recall_recent(limit)` — newest-first N engrams.
- `AdmissionState::recall_by_id(id)` — exact lookup.
- `AdmissionState::recall_by_keyword(keyword, limit)` — case-insensitive
  substring, newest-first, limit-capped. Empty keyword = empty Vec
  (caller-meant-to-skip semantic, not match-everything).
- `AdmissionState::recall_by_origin_kind(kind, limit)` — filter by
  Chat / Airc / Tool / SelfReflection.
- `EngramOriginKind` discriminator enum + `From<&EngramOrigin>` impl —
  exhaustive match means new origin variants force compile-time update.

- `cognition/recall-engrams` IPC handler — kind=recent|by_id|by_keyword|by_origin
  + standard params. Returns `{ engrams, count }` JSON. Defaults to
  kind=recent + limit=10.

What this PR does NOT ship (deferred):

- ORM persistence (PR-6) — engrams still in-memory; queries hit the Vec.
  API stays the same when the backing store swaps.
- Embedding-based / semantic recall (PR-7+) — keyword is substring only.
- Pagination cursors — limit is the only knob; recall_recent doesn't
  expose offset (assumption: callers want the most recent slice).

Tests: 15/15 admission_state pass (was 9, +6 new):
- recall_recent_returns_newest_first
- recall_recent_respects_limit_above_and_below_count
- recall_by_id_finds_known_returns_none_unknown
- recall_by_keyword_case_insensitive_newest_first_with_limit
- recall_by_origin_kind_filters_to_requested_variant
- engram_origin_kind_covers_all_origin_variants (compile-time exhaustive)

Card: continuum#1162. Closes the engram thread substrate (PRs 1-5 +
fix #1157 all merged on canary). The next slice is ORM persistence
(PR-6) or TS-side wiring of the cognition/admit + cognition/recall
handlers from the chat path (separate slice).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply merged commit 92f8109 into canary May 14, 2026
3 checks passed
@joelteply joelteply deleted the feat/engram-recall-surface branch May 14, 2026 12:50
@joelteply
Copy link
Copy Markdown
Contributor Author

Quick review (claude tab #2). Reviewed PR-2/3/4 in this lane; PR-5 closes the read side cleanly.

Architecture — consistent with PR-4

recall_* surface (recent/by_id/by_keyword/by_origin_kind) is the right minimal API for v1. Returns Vec<Engram> (or Option for by_id), newest-first documented + tested, edge cases handled (empty keyword + limit=0 both early-return without scanning).

EngramOriginKind discriminator + From<&EngramOrigin> impl is the clean way to let callers filter by variant without touching the variant-specific payload. Standard pattern, low cost.

IPC handler dispatches on kind parameter with the right defaults (kind=recent + limit=10), validates the kind enum + origin enum strings with explicit error lists, and uses usize::try_from(limit_u64) to defend against u64 overflow on 32-bit. Clean.

Things to consider (none blocking)

  1. No upper-bound on limit parameter. usize::try_from catches overflow, but limit=999_999_999 (just under usize::MAX on 64-bit) would still allocate a massive Vec on a recall_recent call. A sanity cap — limit.min(MAX_RECALL_LIMIT) with MAX_RECALL_LIMIT = 10_000 or similar — bounds the worst-case allocation. Nit; the current limit-clamping at "available" via .take(limit) means the returned Vec can't exceed engrams.len(), but the call itself still does an unbounded Vec::new() allocation if a caller asks for limit=u64::MAX. PR-6 should consider this when ORM-backed queries land (an ORM LIMIT 999_999_999 is its own foot-gun).

  2. recall_by_keyword linear scan + per-engram to_lowercase() — for post-PR-6 ORM with thousands of engrams, linear scan is too slow AND to_lowercase() allocates per call per engram. Comment correctly defers indexing to PR-6. For v1 this is fine; just flagging that the per-engram lowercase allocation is on the same hot path as the not-yet-fixed content_hash_sha256 allocation pattern — both worth a sweep when the persistence layer lands.

  3. IPC error format still flattens typed errors to strings — same observation from my PR-4 review (Err(format!("unknown recall kind '{other}'..."))). PR-5+ JSON-discriminant work referred to in PR-4's TODO covers this; just confirming consistency.

  4. No tests for IPC handler error paths (unknown kind, unknown origin kind, missing required params). The 15 admission_state tests cover the recall fns; the IPC handler dispatch + validation is untested. Consider adding 2-3 IPC-layer tests in a follow-up: known kinds dispatch correctly, unknown kind returns the canonical error list, missing required params (e.g. by_id without id) error cleanly.

What I particularly like

  • EngramOriginKind exists as a separate type rather than overloading EngramOrigin discriminant access. Makes recall_by_origin_kind's signature read self-documenting.
  • Empty-keyword early-return with explicit comment "Avoids the gotcha where every engram contains the empty string." Saves a future reader from rediscovering the bug.
  • Helper admit_n_distinct in tests keeps test setup tight + reusable. Same shape as the analogous helpers in admission_state.rs from PR-4. Consistency.
  • Test names + "What this catches" preambles continue the pattern from PR-2/3/4. Reads as protect-this-contract.

Recommendation

LGTM to merge (already merged per the broadcast). The 4 nits are PR-6 territory or comment additions, none blocking.

The engram lane (#1129#1134#1143#1155#1163#1161 fix) reads as one coherent design across 5 substantive PRs in <24h. Architecture continuity is the win — each PR's review is cheap because the shape was right at PR-1.

PR-6 (ORM persistence) is the next-and-last for the engram thread per the broadcast. Pacing: with admit_inbox + recall both reachable from TS, the in-memory store is enough to build the first persona-with-real-memory demo even before persistence lands.

joelteply added a commit that referenced this pull request May 14, 2026
…#1185)

Per task #71 — survey of every .json under src/system/recipes/.

Findings: the 28 split into 3 pipeline shapes (15 static-view, 10
single-persona-chat, 1 full multi-persona) plus 2 outliers (gan,
academy-training). The 10 single-persona-chat are missing 6 steps
that multi-persona-chat has (loop-risk, fast-respond, training-mode,
record-interaction, chat/send, cooldown). NO recipe currently
integrates the engram admission gate shipped on canary in #1129/
#1134/#1143/#1155/#1163.

5 identified gaps with concrete next-sprint cards:
1. Engram integration in Shape B + C (11 recipes need cognition/
   admit-inbox-message + cognition/recall-engrams)
2. Resolve academy-training half-migrated state
3. Document gan orphan intent
4. Shape B → Shape C decision (or shared inheritance)
5. version field discipline across all 28

Pure docs PR. Output at docs/cognition/RECIPE-AUDIT-2026-05-14.md.

Closes #71.

Co-authored-by: Test <test@test.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant