Give the host every read it still runs raw SQL for

[YellowSnnowmann]

Merge order — this PR is second. It vendors tinyhumansai/tinycortex#157 and does not compile without it. Merge #157 first, re-point vendor/tinycortex at the merge SHA, then merge this. tinymemory v1.5.0 is cut after that, and only then can openhuman#5560's host work begin — the host pins a released, digest-verified artifact, so a member that is merged but unreleased is a runtime UnknownMethod, not a compile error.

Depends on tinyhumansai/tinycortex#157. The gitlink here points at that branch's head; it needs re-pinning to the merge SHA once #157 merges.

Why

openhuman#5560's goal is zero tinymemory_core:: references in the host. What was left after the routing passes was not routing — it was six reads the contract could not express, so five handlers kept a SQLite handle purely to answer them. These are those six.

What was added

MemoryChunks::count_chunks(query, scope) — the unpaged total a paging caller needs. The filter is the listing's own: tinycortex#157 factors the WHERE-clause construction into append_filters and the count reuses it verbatim, appending no LIMIT. A count and the page it labels cannot drift, because they are the same predicate.

Deliberately not defaulted to list_chunks(..).len(). That compiles, passes a naive test, and silently returns the driver's row cap as if it were the total.

Three MemoryEntities members returning what the occurrence index actually holds (entity_id, kind, surface, mention count), plus the entity→chunk-ids lookup that had no member at all. entities() was not widened to fit: it is namespace-scoped and hotness-ranked, and the index is neither. Bending one member to serve both would have made its documented contract a lie — two honestly-named members beat one fudged one.

Two MemoryTree members — summary_forest and recent_leaves — which walk the sealed forest the tree family could describe one node at a time but never enumerate.

Two verdicts, no member

• The doctor's chunk count is already store_stats. Its blocker is that run is sync-only while the member is async — a host restructure, not a contract gap. Adding a sync duplicate would have been a member nobody needed.
• The obsidian registry is host filesystem policy that happens to live in the engine crate, like redact and safety before it. The host can own it.

CONTRACT_VERSION (2, 2) → (3, 0)

Every member here lands on a family a driver may already advertise, and negotiation is family-granular. A driver advertising Chunks at 2.2 would be bound and then asked for a method it has never heard of; the major half refuses that bind instead of discovering it at the call. That is what the rule in version.rs says, in those words.

Worth stating plainly: #85/#86/#89/#90 also added methods to advertised families and stayed on the minor half. That was wrong by this rule. Nothing was bound across the gap because those releases and their hosts moved in lockstep, but it is drift, not precedent — and this PR is the one that stops compounding it.

Consequence: the host must re-pin to the release carrying this. It cannot stay on v1.4.0.

Validation

• METHODS 99 → 103, registered in all four places, verified by sequence comparison rather than by eye
• cargo check --workspace --all-targets clean; module workspace clean
• cargo clippy --all-targets --all-features -- -D warnings — both workspaces
• cargo test --all-features — no failures; conformance 20 passed, module lib 49 passed
• Loader E2E against the real dlopen'ed release cdylib at 103 members: every_declared_method_is_actually_routed, the_manifest_declares_every_method_the_module_serves, the_module_advertises_the_complete_tinymemory_api, stateful_optional_families_round_trip_over_the_bus, query_and_maintenance_families_dispatch_typed_requests — all pass

Known, not fixed here

count_chunks alone does not fully unblock OpenHuman's chunk-list handler: its filter also needs entity_ids (an index join), set-valued source_kinds/source_ids, and a content LIKE scan, none of which ChunkQuery expresses. And the paged row carries lifecycle_status / content_path / has_embedding, which Chunk does not — ChunkDetail has all three but costs 4 queries + a vault read per chunk, so calling it per row would be worse than the SQL it replaces. Closing that needs a batched detail listing, which is a second feature rather than a count.

One finding worth flagging: the column the host reads for has_embedding is dead. Every engine write path goes to the mem_tree_chunk_embeddings sidecar, so mem_tree_chunks.embedding is never written and that flag is effectively always false today. Whatever member ends up feeding it will flip it to true on every embedded chunk — a visible product change, not a routing swap.

---

Update — the remaining gaps for openhuman#5560 are closed here

The PR started as six members. Finishing the survey turned up four more the host still runs raw SQL for, plus two engine-owned runtime behaviours that have no owner at all once the host deletes its in-process engine. Both halves are now in this branch, because splitting them would have deadlocked the release behind a second PR.

Contract

ChunkQuery grows six filters — ids, source_kinds, source_ids, entity_ids, entity_kinds, content_contains — every one #[serde(default, skip_serializing_if = ...)], so an older peer on either side of the boundary neither sends nor misses them. Four trait members join them, all defaulted to Unsupported so no existing driver stops compiling: MemoryChunks::{list_chunk_details, source_totals}, MemorySourceSink::forget_matching, MemoryMaintenance::purge_all.

ChunkListRow is deliberately ChunkDetail without its body. ChunkDetail::body's own docs promise that None means the vault read failed — a listing can only honour that by reading every file or by lying, so it does neither.

MemoryEntities::chunk_entities is amended rather than added: it now takes a batch of chunk ids plus an optional kind filter and answers ChunkEntityOccurrence rows that each name their own chunk. Widening it is legitimate only because it has never shipped — it was added on this branch — and its position in the member sequence is unchanged, which is what the drift assertion pins. The four new methods are registered in all four places that must agree (interface impl, module manifest, tinymemory_bus::METHODS, the E2E's expected set), in declaration order, because that test compares sequences rather than sets.

Runtime — the part that is not a member

The module now starts the engine's queue worker pool itself. Until now the only queue::start in any tree was the host's, against the second in-process engine the host also boots. A host that deletes that engine — the entire point of loading this module — silently turns flush_pending, retry_failed, the re-embed backfill and the ingest path's extract_chunk into no-ops. Ingestion keeps reporting success and the content is simply never indexed. That failure mode is why this is in the same PR as the members.

Two degradations come with it, documented rather than hidden: the pool runs unthrottled, because the scheduler gate is a host seam no bus interface serves, and its shutdown hook is dropped, so locks held at exit are reclaimed by lease expiry on the next start instead of released cleanly. Closing either properly needs a SchedulerGate bus interface, which is separate work. claim_queue_pool exists because queue::start's Once is process-global while a pool is bound to one workspace: a second workspace in one process is not a second pool, it is a store with nothing draining its queue, and that now logs an error instead of being invisible.

tinycortex/contacts replaces tinycortex/people on the module's dependency, since upstream declares contacts = ["people", ...]. Without it SeedFromAddressBook compiles down to the stub returning an empty contact list, so a refresh reports success and imports nobody — the failure mode that looks like an empty address book rather than a missing feature. The four objc2 crates behind the gate sit under a macOS target table upstream, so Linux and Windows artifacts still compile none of them.

The E2E's SeedFromAddressBook assertion now accepts a platform permission denial: with contacts on, that call reaches the real address book, and on macOS that is a per-application privacy grant a test runner may not hold. What is asserted is that the call routes and returns under a contract error name — never that the host granted access.

Verification

Both workspaces cargo check --all-targets; cargo test --workspace (1919 pass) and --all-features (1781 pass); the full CI feature matrix, all 13 rows; clippy -D warnings with --all-targets --all-features on the root and the module workspace; cargo fmt --check on both; cargo doc --no-deps --all-features under -D warnings; cargo deny check; engine-containment.sh and dependency-budget.sh; production-source coverage 81.89% and module coverage 86.81% lines, both over the 80% gate; the --locked release cdylib build; and the loader E2E, all 12 cases, one process per test, against that release artifact.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fd8ce38-9964-4b6f-b0cb-9c2957b68456

Warning

Your free Security trial is over. An organization admin can activate billing to continue.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[tinysweeper]

tinysweeper found nothing blocking. Approving.

             $0.0858 · 522,922 in / 11,492 out · 53,330 cached (10%) · deepseek/deepseek-v4-flash, openrouter/openai/text-embedding-3-small, z-ai/glm-5.2 · 811 embedded
critique:    $0.0157 · 222,960 in / 1,877 out  · 17,815 cached (8%)  · deepseek/deepseek-v4-flash, z-ai/glm-5.2
security:    $0.0211 · 221,199 in / 2,088 out  · 35,515 cached (16%) · deepseek/deepseek-v4-flash, z-ai/glm-5.2
tests:       $0.0468 · 41,116 in  / 6,658 out  · 0 cached (0%)       · z-ai/glm-5.2
description: $0.0019 · 34,021 in  / 81 out     · 0 cached (0%)       · deepseek/deepseek-v4-flash

[tinysweeper]

How this change flows

1 changed behaviour across 8 relationships. 6 surrounding behaviours are shown (60 graph nodes walked). 40 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["node_id_to_path<br/>changed"]:::changed
  n1["call"]:::impacted
  n2["Result"]:::impacted
  n3["join"]:::impacted
  n4["portability_and_lifecycle_round_trip"]:::impacted
  n5["provider_config"]:::impacted
  n6["iter"]:::impacted
  n0 -->|calls| n3
  n1 -->|uses| n2
  n3 -->|calls| n6
  n4 -->|calls| n1
  n4 -->|tests| n1
  n4 -->|uses| n2
  n5 -->|calls| n3
  n5 -->|tests| n3
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff

Loading

_{Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.}