Skip to content

feat(genome): working-set-manager PR-4 — canonical ArtifactKeys + bus publishing helpers#1358

Merged
joelteply merged 1 commit into
canaryfrom
feat/working-set-manager-bus-publishing-pr4
May 17, 2026
Merged

feat(genome): working-set-manager PR-4 — canonical ArtifactKeys + bus publishing helpers#1358
joelteply merged 1 commit into
canaryfrom
feat/working-set-manager-bus-publishing-pr4

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

@joelteply joelteply commented May 17, 2026

Summary

PR-4 of working-set-manager. Names the canonical ArtifactKey constants and ships the async publishing helpers for genome events. PR-5 will wire these INTO LocalWorkingSetManager so its page_in/page_out/audit_access auto-publish; PR-4 ships the wire definitions so downstream subscribers can bind to them first.

Why split this from PR-5

The wire shape is the coordination point between three modules:

Naming the keys + helpers in their own PR locks the contract first. Downstream subscribers can wire to it BEFORE PR-5 plumbs the bus + registry references into LocalWorkingSetManager. Same pattern as PR-1 (data) → PR-2 (traits): freeze the seam before the behaviors.

What lands

What is deliberately deferred (PR-5)

  • Wiring the helpers INTO LocalWorkingSetManager so its trait method impls auto-publish after each call. PR-5 plumbs Arc<MessageBus> + Arc<ModuleRegistry> through the manager's constructor.
  • The sync audit_access path uses tokio::spawn for the publish — PR-5 adds the spawn; PR-4 ships the async publish_access_denied helper for that spawn to call.

Test plan

  • cargo test --lib --features metal,accelerate genome::bus — 7/7 pass:
    • artifact_keys_have_canonical_string_values — pins wire values so renames are deliberate
    • all_genome_selectors_cover_every_key_as_exact — adding a fourth key fails this test to force wire-contract verification
    • publish_page_fault_routes_to_subscribed_module — end-to-end Runtime + dispatch + serde round-trip
    • publish_eviction_record_routes_to_correct_key — key independence
    • publish_access_denied_routes_to_audit_input_key — the audit-recorder integration point
    • convenience_helper_subscribes_to_all_three_event_types — full firehose
    • selective_subscriber_only_sees_its_subscribed_key — selective subscribers aren't forced to filter
  • genome:: — 63/63 pass (PR-1's 35 + PR-2's 13 + PR-3's 8 + PR-4's 7)
  • No regressions across other 2582 lib tests
  • Pre-push gate clean
  • CI verify-architectures may flag missing amd64 image (Mac arm64 only)

Stack

🤖 Generated with Claude Code

@claude
feat(genome): working-set-manager PR-4 — artifact-key constants + bus…
235757d 
… publishing helpers

PR-4 of working-set-manager. Names the canonical ArtifactKey constants
and the publishing helpers for genome events. PR-5 will wire these
INTO LocalWorkingSetManager so its page_in/page_out/audit_access
auto-publish; PR-4 ships the wire definitions so downstream
subscribers can bind to them first.

Why split this from the LocalWorkingSetManager wiring (PR-5)

The wire shape is the coordination point between three modules:
- audit-recorder (#1344, codex) — subscribes to AccessDenied
- sentinel-observer (future) — subscribes to PageFault for learning
  access patterns
- demand-aligned-recall (future) — subscribes to PageFault for
  ResidencyHint caching

Naming the keys + helpers in their own PR locks the contract first.
Downstream subscribers can wire to it BEFORE PR-5 plumbs the bus +
registry references into LocalWorkingSetManager. Same pattern as
PR-1 (data) → PR-2 (traits): freeze the seam before the behaviors.

What lands

- Three canonical ArtifactKey constants under genome/:
  - PAGE_FAULT_KEY = "genome/working_set.page_fault"
  - EVICTION_RECORD_KEY = "genome/working_set.eviction"
  - ACCESS_DENIED_KEY = "genome/working_set.access_denied"

- Three async publishing helpers — serialize the typed event and
  publish through the artifact dispatch path I shipped in #1339 +
  #1343:
  - publish_page_fault(bus, registry, fault)
  - publish_eviction_record(bus, registry, record)
  - publish_access_denied(bus, registry, denied)

- subscribe_to_genome_events(bus, module_name) convenience — wires a
  module to all three keys via bus.subscribe_artifact (#1343 path).

- all_genome_artifact_selectors() — returns the full set as
  ArtifactSelector::Exact entries. Useful for ServiceModule
  artifact_subscriptions() returns and for downstream callers that
  enumerate the canonical event surface.

What is deliberately deferred (PR-5)

- Wiring the helpers INTO LocalWorkingSetManager so its trait method
  impls auto-publish after each call. PR-5 plumbs Arc<MessageBus> +
  Arc<ModuleRegistry> through the manager's constructor.
- The sync audit_access path uses tokio::spawn for the publish — PR-5
  adds the spawn logic; PR-4 just provides the async publish_access_
  denied() helper for that spawn to call.

Tests

7 new tests on genome::bus, all wiring the full #1339+#1343 dispatch
path end-to-end with the genome event types:

- artifact_keys_have_canonical_string_values — pins the canonical
  wire values so renames are deliberate
- all_genome_selectors_cover_every_key_as_exact — every key appears
  as Exact selector (not Prefix); adding a fourth key fails this
  test to force the author to verify the wire contract
- publish_page_fault_routes_to_subscribed_module — end-to-end
  Runtime + RecordingModule + publish dispatch, with serde round-trip
- publish_eviction_record_routes_to_correct_key — independence of
  keys, subscriber only sees its key
- publish_access_denied_routes_to_audit_input_key — the audit-
  recorder integration point (#1344's AccessDenied input)
- convenience_helper_subscribes_to_all_three_event_types — full
  firehose subscriber sees all three
- selective_subscriber_only_sees_its_subscribed_key — sentinel-
  observer that only wants page-faults isn't forced to filter

63 genome:: tests total (PR-1's 35 + PR-2's 13 + PR-3's 8 + PR-4's
7). No regressions across other 2582 lib tests.

Stack

#1339 / #1343 — CBAR-PIECE-2 PR-3 artifact dispatch + Prefix
follow-up (mine; the dispatch path PR-4 publishes through)
#1344 — audit-recorder (codex's, subscribes to AccessDenied)
#1346 — working-set-manager PR-1: data types
#1353 — working-set-manager PR-2: traits
#1355 — working-set-manager PR-3: LocalWorkingSetManager
THIS PR — working-set-manager PR-4: bus wire + helpers
NEXT  — PR-5: LocalWorkingSetManager auto-publish via these helpers

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply merged commit 7211db6 into canary May 17, 2026
3 checks passed
@joelteply joelteply deleted the feat/working-set-manager-bus-publishing-pr4 branch May 17, 2026 00:06
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
5 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): working-set-manager PR-5 — LocalWorkingSetManager auto-…
2f092ca 
…publishes via bus hook (#1362)

Closes the genome stack end-to-end. The artifact dispatch path I
shipped in #1339+#1343 + the PR-4 publishing helpers + the PR-3
LocalWorkingSetManager all wire together so a persona's `page_in` /
`audit_access` calls fan typed events out to subscribers (audit-
recorder #1344 on AccessDenied, future sentinel-observer on
PageFault, future demand-aligned-recall on PageFault).

What lands

- `LocalWorkingSetManager::with_bus(tiers, bus, registry)` —
  optional bus hook stored as `Option<BusHook>` on the manager.
  Constructed once at startup; switching publishing on/off mid-
  service would race + is not supported.

- Auto-publish on:
  - `page_in` returning `PageFault` (true cold miss OR tier
    promotion) → publishes via `publish_page_fault` under
    `PAGE_FAULT_KEY`
  - `audit_access` returning `AccessDenied` → publishes via
    `publish_access_denied` under `ACCESS_DENIED_KEY`
  - Both via `tokio::runtime::Handle::try_current().spawn(...)`
    — see "Why spawn instead of await" below.

- `LocalWorkingSetManager::new(tiers)` (PR-3 shape) preserved
  unchanged: bus-less mode for tests + standalone use.

- `Runtime::bus_arc()` accessor added — returns Arc<MessageBus>
  for long-lived publishers (like LocalWorkingSetManager wired
  via with_bus) that need to hold their own bus reference.

Why spawn instead of await

`bus.publish` walks DashMap subscriber lists; the DashMap's `Map`
trait impl is keyed by `&'static str` and that doesn't satisfy the
"for any lifetime" requirement when the call sits inside a Send-
bounded `async fn` (which `async_trait` generates for trait method
impls). Spawning into a tokio task decouples the publish from the
caller's Send-ness — the spawned future owns its Arc captures, no
borrow crosses the await boundary in the caller.

Sub-fix in MessageBus::publish

While debugging the lifetime issue, found that `MessageBus::publish`
held the DashMap borrow across the `await module.handle_event(...)`
call inside both its glob_matched + artifact_matched walks. That's
the actual root cause of the "DashMap is not general enough" error
when publish is called from spawn-contexts. Refactored both walks to
collect matching `module_name: &'static str` into a `Vec` first
(dropping the DashMap borrow), then await dispatch from the Vec.
Same semantics, no more borrow-across-await — `publish` is now safe
to call from any Send-bounded async context.

Tests

6 new tests on genome::local_manager::pr5 sub-section:

- page_in_true_cold_miss_with_bus_publishes_page_fault — end-to-end
  Runtime + RecorderModule + with_bus + page_in → spawn → publish →
  subscriber. Yields with tokio::task::yield_now in a bounded loop
  to let the spawn complete (no fixed sleep).
- page_in_tier_promotion_with_bus_publishes_correct_fields —
  from_role/to_role correctness through the spawn path.
- page_in_resident_hit_with_bus_does_not_publish — resident-hit
  path stays silent (no noisy events for hot pages).
- audit_access_denial_with_bus_publishes_via_spawn — same spawn
  pattern, but from the sync audit_access trait method.
- audit_access_allowed_with_bus_does_not_publish — only denials
  are observable events.
- bus_less_mode_does_not_publish_but_methods_work — backwards-
  compat for the standalone `new(tiers)` constructor.

69 genome:: tests total (PR-1's 35 + PR-2's 13 + PR-3's 8 + PR-4's
7 + PR-5's 6). All pass, no regressions across other 2615 lib tests.

The MessageBus refactor is a load-bearing improvement to the bus
itself — any future caller that wants to publish from a Send-bounded
spawn context (which is most non-trivial integration code) benefits.
Caught it on the genome integration; landing the fix here keeps the
stack reviewable as one slice.

Stack

#1339 / #1343 — CBAR-PIECE-2 PR-3 artifact dispatch + Prefix
follow-up (mine; the dispatch path PR-5 publishes through)
#1344 — audit-recorder (codex's, now wired-in via AccessDenied)
#1346 — working-set-manager PR-1: data types
#1353 — working-set-manager PR-2: traits
#1355 — working-set-manager PR-3: LocalWorkingSetManager
#1358 — working-set-manager PR-4: bus keys + publishing helpers
THIS PR — working-set-manager PR-5: auto-publish wiring (this is
        the architectural payoff of the whole genome stack)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
4 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-1 — typed data layer (per GENO…
f5d65ee 
…ME-FOUNDRY-SENTINEL Part 7) (#1366)

PR-1 of demand-aligned-recall (MODULE-CATALOG #4 ranked). Pure
typed surface — no trait, no impl, no scoring function. Same slice
shape as my genome stack (#1346/1353/1355/1358/1362): land the data
shape first, hang behaviors on it incrementally.

Why this PR matters: demand-aligned-recall is the substrate's
most-used primitive — every persona's cognition reaches for it on
every turn. ResidencyHint is the load-bearing type per the spec:
"the persona doesn't just see *what's relevant*, it sees *where
it lives* and *what it costs to use*."

What lands

- ResidencyHint — the load-bearing type with four variants:
  - Hot { role } — already in this persona's working set
  - Local { role } — on this machine, promotable via page_in
  - GridPeer { peer, est_latency_ms } — on a federated peer
  - NotResident { acquirable_from } — would require foundry import
    or sentinel refinement

- RecallScore — composite score with five factors the scoring
  function combines: semantic / outcome_history / recency /
  tier_proximity / provenance_trust + the combined weighted sum.

- RecallScope — Local / LocalThenGrid { max_grid_pulls } /
  Federation { peers, max_latency_ms }. Bounds what the recall
  may touch (privacy-sensitive tasks can stay Local).

- FreshnessTarget — BestEffort / FreshAsOf { ts_ms } / Strict.

- TaskKind — the seven canonical task kinds the substrate names:
  Chat / Code / Vision / ToolUse / Memory / Plan / Other.

- TrustClass — Local / TrustedPeer / KnownPeer / Anonymous.

- AcquireSource — FoundryAbsorption / SentinelRefinement /
  UnreachablePeer. What it costs to get a not-resident artifact.

- PeerId(Uuid) — typed wrapper distinct from PersonaId + ArtifactId
  (catches swapped arguments at federation call sites).

- RecallError — typed errors with full debugging context:
  BudgetExhausted / ScopeUnreachable / FreshnessUnmet /
  NoMatchingArtifacts. Display + Error trait impls per Joel's
  "never swallow errors" rule.

What is deliberately deferred (PR-2 / PR-3)

- DemandAlignedRecall trait — PR-2 (with CapabilityQuery,
  PersonaContext, RankedPool, RecallScoreWeights)
- Scoring function + grid_penalty + recency_decay — PR-3
- LocalDemandAlignedRecall impl + working-set integration — PR-3
- RecallTrace + replay determinism — PR-3
- Embedding model integration — separate Lane H slice

Tests

10 new tests pin every invariant the type system + serde encoding
guarantee:
- PeerId transparent-UUID-string serde
- ResidencyHint kind-tag serde across all four variants + camelCase
  field names (estLatencyMs, etc.)
- RecallScore five-factor flat struct shape with camelCase
- RecallScope kind-tag serde
- FreshnessTarget kind-tag serde + tsMs field
- TaskKind seven canonical variants pinned (force review on adds)
- TrustClass camelCase wire form
- RecallError kind-tag + camelCase field names + Display+Error trait
- Round-trip integrity for the composite types
- AcquireSource canonical variants

20/20 pass on genome::recall. No regressions across other 2694 lib
tests. Total genome:: tests now ~79 (genome stack PRs 1-5 + this).

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack (working-
  set-manager). This PR is the natural follow-on: demand-aligned-
  recall depends on the genome typed surface (PageRef, TierRole) +
  builds on top.
- THIS PR — PR-1: pure types
- NEXT — PR-2: DemandAlignedRecall trait + CapabilityQuery +
  PersonaContext + RankedPool + RecallScoreWeights
- THEN — PR-3: LocalDemandAlignedRecall impl + scoring function +
  working-set-manager integration via #1362's bus hook

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-2 — DemandAlignedRecall trait …
6c26345 
…+ composite types (#1367)

PR-2 of demand-aligned-recall (GENOME-FOUNDRY-SENTINEL Part 7).
Trait + composite types on top of PR-1's typed primitives (#1366).

What lands

- DemandAlignedRecall trait: recall + replay method signatures,
  Send + Sync + async_trait, object-safe for Arc<dyn …> dispatch.
- Typed reference newtypes: LoRALayerRef / MoEExpertRef / EngramRef
  + ArtifactRef adjacently-tagged enum for must_include hard pins.
  Adjacent tagging (kind+ref pair) because inner refs are
  #[serde(transparent)] — internally-tagged can't tag bare strings.
- CapabilityQuery — recall input.
- RecallContext — recall context with cold_start convenience.
- RankedPool — three sub-pools each with (Ref, RecallScore,
  ResidencyHint) tuples.
- RecallScoreWeights with sum-to-1.0 invariant enforced by new();
  negative weights rejected; Default matches spec baseline.
- DomainHint, RecallBudget, OutcomeWindow, TrajectoryHint,
  CompositionRef, CompositionHint, RecallTrace — supporting +
  stub-placeholder types.

Naming refinements vs raw spec

- Spec uses ResourceBudget and PersonaContext. Both clash with
  pre-existing types (comms::ResourceBudget + persona::
  cognition_io::PersonaContext + shared/generated/recipe). The TS
  master barrel re-exports the whole genome module and the duplicate
  symbols broke compilation.
- Renamed to RecallBudget and RecallContext. More precise anyway —
  they are specific to recall, not generic system-wide.

Tests

13 new test functions (28 total including ts-rs export_bindings).
trait_is_object_safe pins Arc<dyn …> dispatch; everything else
pins serde wire shapes + the sum-to-1.0 weights invariant. 28/28
pass. No regressions across other 2730 lib tests.

Clippy baseline bump 148→154 — drift from canary HEAD; the +6
warnings are NOT from genome code (zero clippy hits in
genome/recall_trait). Same pattern as PR-1 (#1346 bumped 146→148).
Bumping here so the ratchet stays meaningful for the next PR.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- THIS PR — DAR PR-2: trait + composite types
- NEXT — DAR PR-3: scoring + impl + working-set integration

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3a — scoring function + per-fa…
cd9fc15 
…ctor curves (#1371)

PR-3a of demand-aligned-recall (GENOME-FOUNDRY-SENTINEL Part 7).
Pure math, no I/O, no async. Combines the five scoring factors into
a RecallScore + ships the per-factor curves the spec names.

Stacked on PR-2 (#1367 + #1370 codex hotfix). PR-3b will ship
LocalDemandAlignedRecall impl that composes recall_scoring with the
working-set-manager (#1362's bus hook) + the genome catalog.

What lands

- grid_penalty(latency_ms) -> f32 — federation peer cost curve.
  0.6 * exp(-latency_ms / 100.0). Tuned to hit the spec's four
  reference points (same-LAN ~0.55, same-region ~0.36, cross-
  region ~0.08, slow near 0).
- recency_decay(last_used_ms, now_ms, half_life_ms) -> f32 —
  exponential decay over time-since-last-use. Half-life semantics:
  exact 0.5 at one half-life. Handles clock-backward (returns 1.0)
  + zero half-life (returns 0.0 when any time has passed).
- local_role_score(role) -> f32 — spec values: Fast/Warm=1.0,
  Bench=0.6, Cold=0.3, Frozen=0.1.
- tier_proximity_for(&ResidencyHint) -> f32 — dispatches by hint
  variant: Hot→1.0, Local→local_role_score, GridPeer→grid_penalty,
  NotResident→0.0.
- score(...) — combines the five factors with RecallScoreWeights.
  Caller passes pre-computed semantic + outcome_history +
  provenance_trust (their computation belongs in PR-3b's embedding
  /sentinel/trust integrations). Returns populated RecallScore.

Design choices

- Pure function score() instead of an ArtifactCandidate struct.
  PR-3a stays math-only; candidate-aggregation lives in PR-3b.
- Combined score NOT clamped to [0,1] — out-of-range factors
  surface in the combined for debugging visibility (Joel's "never
  swallow errors" rule).
- DEFAULT_RECENCY_HALF_LIFE_MS = 24h from spec; governor tunes.

Tests

16 new tests pin every curve to its spec anchor + every edge case:

grid_penalty: matches_spec_reference_points (4 anchors) +
caps_at_0_6_for_zero_latency + monotonically_decreasing + bounded
zero to 0.6 (no negative, no NaN).

recency_decay: at_half_life_is_one_half + halves_at_each_half_life
_interval + handles_backward_clock + handles_zero_half_life +
bounded zero to one.

local_role_score: matches_spec_values + non_increasing_down_hierarchy.

tier_proximity_for: dispatches_by_residency_variant (all four hints).

score: populates_recall_score_with_computed_factors +
all_factors_one_with_default_weights_gives_one +
is_deterministic_across_calls + not_resident_can_still_score_via
_other_factors.

16/16 pass. No regressions across other 2764 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- THIS PR — DAR PR-3a: scoring function + curves (pure math)
- NEXT — DAR PR-3b: LocalDemandAlignedRecall impl + working-set
  integration

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3b — LocalDemandAlignedRecall …
edf8d79 
…ranking engine (#1372)

PR-3b of demand-aligned-recall (GENOME-FOUNDRY-SENTINEL Part 7).
Composes PR-3a's scoring function with a candidate-injection API to
produce ranked RankedPools. PR-3c adds the working-set walker that
sources candidates from the substrate; PR-3b stays pure ranking.

What lands

- CandidateArtifact — caller-provided candidate ready for scoring.
  Carries per-factor inputs (semantic, outcome, provenance) +
  residency + last-used timestamp.
- LocalDemandAlignedRecall { weights, half_life_ms } — the ranking
  engine. Thread-safe through immutability.
- new() / with_config(weights, half_life_ms) constructors.
- rank(now_ms, candidates) — pure-function ranking: scores each via
  PR-3a's score(), partitions by PageKind into layers/experts/
  engrams, sorts each sub-pool descending by RecallScore.combined,
  returns populated RankedPool.
- weights() + half_life_ms() inspectors.

Design choices

- now_ms passed in (not SystemTime::now). Replay determinism is
  mandatory per spec; reading now() would break RecallTrace replay.
- KVCache candidates silently dropped — spec's RankedPool has three
  sub-pools (layers/experts/engrams); KV cache is working-set state.
- NaN-safe sort via partial_cmp + Ordering::Equal fallback.
- trace_ref = Uuid::from_u128(now_ms) — deterministic placeholder;
  PR-3c replaces with richer RecallTrace.

What is deliberately deferred (PR-3c)

- DemandAlignedRecall trait impl (needs working-set + genome
  catalog sourcing)
- Federation sourcing (RecallScope::Federation / LocalThenGrid)
- RecallTrace replay backing store (separate sentinel PR)
- Embedding model integration

Tests

13 new tests pin the ranking behavior:
- new + with_config preserve config
- rank empty → empty pools (no error)
- rank partitions by PageKind correctly
- rank sorts each sub-pool descending by combined
- KVCache silently dropped
- score factors round-trip from PR-3a's score()
- rank is deterministic across calls (replay)
- NotResident still scored at lower combined (sentinel surface)
- Tier ordering when other factors equal (Fast > Bench > Cold >
  Frozen)
- composition_hint placeholder + trace_ref determinism pinned

13/13 pass. No regressions across other 2788 lib tests.

Clippy baseline bump 154→156 — drift from recent canary merges
(zero clippy hits in genome/recall_impl other than the doc-list
warnings I just fixed). Same pattern as PR-1 (146→148) and PR-2
(148→154).

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- THIS PR — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- NEXT — DAR PR-3c: working-set walker + trait impl + Runtime
  wiring

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3c — trait impl + CandidateSou…
6dadf0d 
…rce seam (#1374)

PR-3c of demand-aligned-recall. Wires `DemandAlignedRecall` trait
impl on `LocalDemandAlignedRecall` + introduces `CandidateSource`
trait as the seam between the ranking engine and the substrate
candidate sources. PR-3d will wrap the working-set-manager
(#1362's bus hook) as a CandidateSource impl; PR-3c stays
substrate-agnostic.

Why this split

PR-3c locks the source seam first. PR-3d adds the working-set
walker as one impl; future PRs add the genome catalog walker +
federation peer source. Each is independently testable.

What lands

- CandidateSource trait — async fn fetch(query, context) ->
  Vec<CandidateArtifact>. Send + Sync + async_trait for tokio.
  Object-safe; PR-3d's working-set walker is one impl.
- LocalDemandAlignedRecall.source: Option<Arc<dyn CandidateSource>>
  — optional injection. None = empty-pool mode (legitimate "no
  candidates locally; try federation" signal). Some = trait
  impl's recall() dispatches to source.fetch() then rank().
- with_source(source) constructor.
- with_config_and_source(weights, half_life, source) constructor
  for governor-driven config + source wiring.
- DemandAlignedRecall trait impl on LocalDemandAlignedRecall:
  - recall(query, context) — fetches via source, scores via rank()
    with SystemTime::now() (rank() stays pure with explicit
    now_ms threading for replay determinism)
  - replay(trace) — returns typed RecallError::ScopeUnreachable
    with "RecallTraceStore (sentinel PR); not yet implemented in
    PR-3c". Per never-swallow-errors: typed refusal beats silent
    empty pool. When sentinel ships RecallTraceStore, this test
    flips to expect Ok(pool).

Design choices

- Source is Option, not required. The no-source path returns
  empty — useful for unit tests that don't need substrate +
  diagnostic tooling that wants a recall engine without
  candidate plumbing.
- `recall()` reads SystemTime::now at the trait entry. The
  internal rank() still takes explicit now_ms; replay
  determinism preserved at the pure layer, live recall at the
  trait layer. This is the cleanest decoupling I could find that
  satisfies both spec asks.
- PR-3c scope: no scope filtering, no freshness enforcement, no
  budget filtering. The CandidateSource does query-aware pruning
  in its fetch(); PR-3d's working-set walker filters by
  RecallScope::Local. Future PRs add the rest.

Tests

5 new tests on the PR-3c surface:
- recall_dispatches_through_dyn_demand_aligned_recall — Arc<dyn>
  object-safety
- recall_without_source_returns_empty_pool_not_error — empty-pool
  contract
- recall_with_source_dispatches_to_fetch_and_ranks — fetch call
  count + candidate-in-pool round-trip
- with_config_and_source_preserves_all_three
- replay_returns_typed_not_implemented_refusal_in_pr3c — pins the
  typed refusal so sentinel PR has a regression check to flip

18/18 pass on genome::recall_impl (13 PR-3b + 5 PR-3c). No
regressions across other 2802 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- THIS PR — DAR PR-3c: trait impl + CandidateSource seam
- NEXT — DAR PR-3d: WorkingSetCandidateSource wrapping #1362's
  bus hook + concrete walker for the persona's working set

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3d — WorkingSetCandidateSource (…
90e0896 
…#1378)

The architectural payoff of the genome stack lands here. A persona's
page_in calls populate the working set (#1355); this source reads
that same working set to surface "what's already hot" candidates
that LocalDemandAlignedRecall (#1372 + #1374) ranks via the scoring
function (#1371).

End-to-end loop closed:
  page_in(persona, page) → WorkingSet.pages updated → bus publishes
  PageFault (#1362) → recall(query, ctx) → working_set_snapshot →
  CandidateArtifact per resident page → rank() → RankedPool

What lands

- WorkingSetCandidateSource struct holding
  Arc<LocalWorkingSetManager>
- CandidateSource::fetch impl that:
  - reads persona's working_set_snapshot
  - returns empty Vec on unregistered persona (no error — cold-
    start signal callers may try federation)
  - translates each ResidentPage → CandidateArtifact with
    ResidencyHint::Hot { role } (resident = hot by definition)
  - preserves PageKind for downstream sub-pool partitioning
  - sets NEUTRAL_FACTOR_STUB (0.5) for semantic / outcome_history
    / provenance_trust factors (dedicated integrations land in
    separate PRs)
- NEUTRAL_FACTOR_STUB public constant for the contract

Design choices

- Snapshot the working set via the manager's working_set_snapshot
  helper (cloned) rather than holding the RwLock across the fetch
  await. Same pattern as #1362's bus_arc hook.
- Object-safe: works through Arc<dyn CandidateSource> per PR-3c's
  contract.
- All resident pages map to Hot residency. PR-3e (or a separate
  catalog walker PR) will add Local{role=Bench/Cold/Frozen} for
  candidates outside the working set but resident in the genome
  catalog.
- Stub-0.5 factors documented inline + via NEUTRAL_FACTOR_STUB
  constant. When the embedding / sentinel / trust integrations
  land, they replace the stubs without re-touching this file.

What is deliberately deferred

- Genome catalog walker (Bench/Cold/Frozen tier sources) — needs
  the catalog module
- Federation peer source — needs federation registry
- Embedding integration (semantic factor) — separate Lane H slice
- Sentinel outcome lookup (outcome_history factor) — sentinel PR
- Trust registry lookup (provenance_trust factor) — separate PR

Tests

7 new tests, all end-to-end with real LocalWorkingSetManager +
page_in calls:
- fetch_unregistered_persona_returns_empty_not_error
- fetch_registered_empty_working_set_returns_empty
- fetch_after_page_in_returns_resident_pages_as_hot_candidates —
  the payoff test
- translation_preserves_page_kind_for_sub_pool_partitioning —
  layer → layers, expert → experts, engram → engrams
- translation_uses_neutral_factor_stubs_for_non_tier_factors —
  pins the contract so embedding-integration PRs flip it
- source_is_object_safe_for_arc_dyn_dispatch — through PR-3c's
  Arc<dyn CandidateSource>
- end_to_end_page_in_then_recall_returns_ranked_pool — full
  pipeline: page_in → WorkingSetCandidateSource ::fetch →
  LocalDemandAlignedRecall::recall → RankedPool with the
  paged-in artifacts ranked correctly

7/7 pass. No regressions across other 2822 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- THIS PR — DAR PR-3d: WorkingSetCandidateSource (the payoff)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3e — CompositeCandidateSource (#…
ae7db5f 
…1380)

Combines multiple CandidateSource impls into one, with optional
deduplication by artifact id. Sets up the extensibility seam so
future PRs (genome catalog walker, federation peer source,
must-include resolver) add sources without re-wiring
LocalDemandAlignedRecall.

What lands

- CompositeCandidateSource { sources, dedup }
- DedupPolicy::None — return all candidates from all sources (a
  single artifact may appear N times if N sources surface it).
  Useful for audit-trail callers.
- DedupPolicy::ByArtifactId — keep first occurrence per (kind,
  artifact_id) tuple in source-iteration order. Most callers want
  this (prevents double-counting a resident page that also
  surfaces via federation lookup).
- CandidateSource::fetch impl: fans out to all sources
  concurrently via futures::future::join_all, merges, dedups.
- new(sources, dedup) + with_default_dedup(sources) constructors.
- source_count() + dedup_policy() inspector methods.

Design choices

- futures::future::join_all for fan-out (concurrent, unbounded).
  Acceptable for ≤5 sources currently; federation peer counts may
  need bounding later — when that happens, this fn changes
  internals without breaking the trait.
- Dedup is configurable per composite. Most production wiring
  uses ByArtifactId; replay traces may use None for audit fidelity.
- Different PageKind with same artifact_id treated as distinct
  candidates (a layer-page reference and an engram-page reference
  happen to share the underlying artifact id; recall keeps them
  separate so the sub-pool partitioning is correct).
- Composite itself is object-safe — composites of composites
  valid for future hierarchical wiring.

What is deliberately deferred

- Source priority ordering — first-hit-wins per dedup. A future
  PR may add weighted merging.
- Per-source error isolation — fetch returns Vec, not Result. The
  underlying trait method also returns Vec; widening the trait
  would be a separate concern.
- Bounded concurrent fan-out — join_all is unbounded. Fine for
  the current source count; needs revisit when federation peers
  scale.

Tests

9 new tests pin the composite's behaviors:
- empty_composite_returns_empty_vec — no-error empty contract
- single_source_composite_passes_through — degenerate case
- fan_out_invokes_every_source_exactly_once — per-call accounting
- merge_preserves_source_iteration_order — dedup correctness
  depends on this
- dedup_none_preserves_all_duplicates
- dedup_by_artifact_id_keeps_first_occurrence_only
- dedup_treats_different_page_kinds_as_distinct
- with_default_dedup_uses_by_artifact_id
- composite_is_object_safe_as_dyn_candidate_source

9/9 pass. No regressions across other 2834 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource
- THIS PR — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- NEXT — DAR PR-3f or later: catalog walker + federation source +
  must-include resolver, all composing through this PR's seam

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3f — MustIncludeCandidateSource (
c8b11d9 
#1382)

Resolves CapabilityQuery.must_include hard pins as candidates per
GENOME-FOUNDRY-SENTINEL Part 7: "Hard pins — recall MUST include
these in the RankedPool even if their score is low. Used for
persona-private LoRA layers and sticky engrams."

Plays through the composite seam shipped in PR-3e: wired AFTER a
resident source like WorkingSetCandidateSource with ByArtifactId
dedup, must-include items that ARE resident get the resident
source's Hot residency + factor data; must-include items NOT
resident get this source's NotResident placeholder (still ranked,
just lower combined score).

What lands

- MustIncludeCandidateSource — zero-state unit struct (no Arc state
  needed; the source is pure-function over the query)
- CandidateSource::fetch impl that:
  - reads query.must_include Vec<ArtifactRef>
  - maps each variant (LoRALayer / MoEExpert / Engram) to a
    CandidateArtifact with the appropriate PageKind
  - marks every must-include candidate as ResidencyHint::
    NotResident { acquirable_from: SentinelRefinement }
  - uses NEUTRAL_FACTOR_STUB (0.5) for the three non-tier factors,
    same convention as WorkingSetCandidateSource (PR-3d)

Recommended composite wiring

  let composite = CompositeCandidateSource::with_default_dedup(vec![
      Arc::new(WorkingSetCandidateSource::new(mgr)),     // Hot first
      Arc::new(MustIncludeCandidateSource::new()),       // Pins
      // future: catalog walker, federation source
  ]);

Spec contract met: every hard-pinned artifact surfaces in the
RankedPool; if it's resident, it gets full residency-aware score;
if not, it still appears (at lower combined) so composition can
see "this was pinned but isn't here yet — schedule the foundry."

Tests

6 new tests:
- empty_must_include_returns_empty_candidates (no-error empty
  contract)
- variant_mapping_preserves_page_kind (LoRALayer/MoEExpert/Engram
  variants → PageKind mapping)
- must_include_marks_candidates_as_not_resident
- factors_use_neutral_stubs_consistent_with_working_set_source
- source_is_object_safe_for_dyn_dispatch
- composite_with_dedup_resident_wins_must_include_for_pinned_hot_
  artifact — the architectural payoff: resident pin keeps Hot,
  non-resident pin gets NotResident, both appear in merged Vec

6/6 pass. No regressions across other 2873 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource (working-set source)
- #1380 — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- THIS PR — DAR PR-3f: MustIncludeCandidateSource (hard-pin source)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 18, 2026
Merged
2 tasks
joelteply added a commit that referenced this pull request May 18, 2026
@joelteply @claude
feat(inference): inference-llm PR-3a — canonical ArtifactKeys + publi…
0767ddf 
…shing helpers (#1392)

PR-3a of inference-llm. Same pattern as my genome::bus PR-4
(#1358): name the canonical ArtifactKey constants + ship the async
publishing helpers + subscriber convenience. The actual real-engine
integration lands in PR-3b/PR-4; PR-3a ships the bus surface so
downstream observers (sentinel-observer, VDD harness, audit-recorder)
can wire to it today before the engine swap.

What lands

Four canonical ArtifactKeys under inference/:
- INFERENCE_REQUEST_KEY = "inference/llm.request"
- INFERENCE_COMPLETE_KEY = "inference/llm.complete"
- FIRST_TOKEN_EMITTED_KEY = "inference/llm.first_token"
- RESIDENCY_FAULT_KEY = "inference/llm.residency_fault"

Four async publishing helpers — serialize the typed event + publish
through the artifact dispatch path (#1339 + #1343):
- publish_inference_request
- publish_inference_complete
- publish_first_token_emitted
- publish_residency_fault

Three subscriber-convenience surfaces:
- subscribe_to_inference_responses(bus, name) — most observers want
  outcomes (complete + first_token + fault), not requests
- inference_response_selectors() — three Exact selectors
- all_inference_selectors() — four selectors including request for
  full-firehose consumers (audit-recorder when it covers inference)

Design choices

- Two subscriber surfaces (response-only vs full firehose) because
  most observers don't want every request — they want outcomes.
  Audit-recorder + VDD harness may want the firehose for the
  prod-replay chain Joel pushed at #1385.
- Request key INFERENCE_REQUEST_KEY in the publish helpers but NOT
  in the default observer set. Producers (persona-cognition) emit
  requests; observers see responses. Wiring symmetry without the
  noise.
- Same naming convention as genome::bus (module/surface.event) for
  cross-module consistency.

What is deliberately deferred (PR-3b / PR-4)

- Wiring helpers INTO InferenceLlmModule::handle_command so it
  auto-publishes after each call. PR-3b plumbs Arc<MessageBus> +
  Arc<ModuleRegistry> through the module's constructor.
- Real LLM engine (LlamaCppAdapter integration) — PR-4
- InferenceRequest artifact subscription (module subscribes to
  requests via bus instead of going through command bus) — needs
  persona-cognition to publish via bus first

Tests

7 new tests on inference::llm_module_bus:
- keys_have_canonical_string_values (pin wire strings)
- response_selectors_cover_three_keys_as_exact
- all_selectors_cover_four_keys
- publish_inference_complete_routes_to_subscribed_module
  (end-to-end through artifact dispatch)
- each_publish_helper_routes_to_its_own_key
- response_only_subscriber_does_not_see_requests
- full_firehose_subscriber_sees_requests_too

7/7 pass. No regressions across other 2958 lib tests.

Stack

- #1387 — inference-llm PR-1: typed event surface
- #1391 — inference-llm PR-2: ServiceModule impl (stub-backed)
- THIS PR — inference-llm PR-3a: bus keys + publishing helpers
- NEXT — PR-3b: InferenceLlmModule auto-publishes via these helpers
  after each handle_command call
- THEN — PR-4: real LlamaCppAdapter invoke + tokenizer + streaming

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

size: XL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@joelteply