Skip to content

feat(genome): demand-aligned-recall PR-1 — typed data layer (Part 7)#1366

Merged
joelteply merged 1 commit into
canaryfrom
feat/demand-aligned-recall-types-pr1
May 17, 2026
Merged

feat(genome): demand-aligned-recall PR-1 — typed data layer (Part 7)#1366
joelteply merged 1 commit into
canaryfrom
feat/demand-aligned-recall-types-pr1

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

Summary

PR-1 of demand-aligned-recall (MODULE-CATALOG #4 ranked, GENOME-FOUNDRY-SENTINEL Part 7). Pure typed surface — no trait, no impl, no scoring function. Same slice shape as my genome stack (#1346/1353/1355/1358/1362).

Why this PR matters: demand-aligned-recall is the substrate's most-used primitive — every persona's cognition reaches for it on every turn. ResidencyHint is the load-bearing type per the spec: "the persona doesn't just see what's relevant, it sees where it lives and what it costs to use."

What lands

  • ResidencyHint — load-bearing type with four variants:
    • Hot { role } — already in this persona's working set
    • Local { role } — on this machine, promotable via page_in
    • GridPeer { peer, est_latency_ms } — on a federated peer
    • NotResident { acquirable_from } — would require foundry import or sentinel refinement
  • RecallScore — composite score with the five factors the scoring function combines: semantic / outcome_history / recency / tier_proximity / provenance_trust + the combined weighted sum.
  • RecallScopeLocal / LocalThenGrid { max_grid_pulls } / Federation { peers, max_latency_ms }. Bounds what recall may touch.
  • FreshnessTargetBestEffort / FreshAsOf { ts_ms } / Strict.
  • TaskKind — seven canonical task kinds: Chat / Code / Vision / ToolUse / Memory / Plan / Other.
  • TrustClassLocal / TrustedPeer / KnownPeer / Anonymous.
  • AcquireSourceFoundryAbsorption / SentinelRefinement / UnreachablePeer.
  • PeerId(Uuid) — typed wrapper distinct from PersonaId + ArtifactId.
  • RecallError — typed errors with full debugging context + Display + Error per Joel's "never swallow errors" rule.

What is deliberately deferred

Test plan

  • cargo test --lib --features metal,accelerate genome::recall — 20/20 pass:
    • PeerId transparent-UUID-string serde
    • ResidencyHint kind-tag serde + camelCase fields
    • RecallScore five-factor flat struct shape
    • RecallScope kind-tag serde
    • FreshnessTarget kind-tag serde
    • TaskKind seven canonical variants pinned
    • TrustClass camelCase wire form
    • RecallError kind-tag + camelCase fields + Display+Error
    • Round-trip integrity for the composite types
    • AcquireSource canonical variants
  • No regressions across other 2694 lib tests
  • Pre-push gate clean
  • CI verify-architectures may flag missing amd64 image

Stack

🤖 Generated with Claude Code

…ME-FOUNDRY-SENTINEL Part 7)

PR-1 of demand-aligned-recall (MODULE-CATALOG #4 ranked). Pure
typed surface — no trait, no impl, no scoring function. Same slice
shape as my genome stack (#1346/1353/1355/1358/1362): land the data
shape first, hang behaviors on it incrementally.

Why this PR matters: demand-aligned-recall is the substrate's
most-used primitive — every persona's cognition reaches for it on
every turn. ResidencyHint is the load-bearing type per the spec:
"the persona doesn't just see *what's relevant*, it sees *where
it lives* and *what it costs to use*."

What lands

- ResidencyHint — the load-bearing type with four variants:
  - Hot { role } — already in this persona's working set
  - Local { role } — on this machine, promotable via page_in
  - GridPeer { peer, est_latency_ms } — on a federated peer
  - NotResident { acquirable_from } — would require foundry import
    or sentinel refinement

- RecallScore — composite score with five factors the scoring
  function combines: semantic / outcome_history / recency /
  tier_proximity / provenance_trust + the combined weighted sum.

- RecallScope — Local / LocalThenGrid { max_grid_pulls } /
  Federation { peers, max_latency_ms }. Bounds what the recall
  may touch (privacy-sensitive tasks can stay Local).

- FreshnessTarget — BestEffort / FreshAsOf { ts_ms } / Strict.

- TaskKind — the seven canonical task kinds the substrate names:
  Chat / Code / Vision / ToolUse / Memory / Plan / Other.

- TrustClass — Local / TrustedPeer / KnownPeer / Anonymous.

- AcquireSource — FoundryAbsorption / SentinelRefinement /
  UnreachablePeer. What it costs to get a not-resident artifact.

- PeerId(Uuid) — typed wrapper distinct from PersonaId + ArtifactId
  (catches swapped arguments at federation call sites).

- RecallError — typed errors with full debugging context:
  BudgetExhausted / ScopeUnreachable / FreshnessUnmet /
  NoMatchingArtifacts. Display + Error trait impls per Joel's
  "never swallow errors" rule.

What is deliberately deferred (PR-2 / PR-3)

- DemandAlignedRecall trait — PR-2 (with CapabilityQuery,
  PersonaContext, RankedPool, RecallScoreWeights)
- Scoring function + grid_penalty + recency_decay — PR-3
- LocalDemandAlignedRecall impl + working-set integration — PR-3
- RecallTrace + replay determinism — PR-3
- Embedding model integration — separate Lane H slice

Tests

10 new tests pin every invariant the type system + serde encoding
guarantee:
- PeerId transparent-UUID-string serde
- ResidencyHint kind-tag serde across all four variants + camelCase
  field names (estLatencyMs, etc.)
- RecallScore five-factor flat struct shape with camelCase
- RecallScope kind-tag serde
- FreshnessTarget kind-tag serde + tsMs field
- TaskKind seven canonical variants pinned (force review on adds)
- TrustClass camelCase wire form
- RecallError kind-tag + camelCase field names + Display+Error trait
- Round-trip integrity for the composite types
- AcquireSource canonical variants

20/20 pass on genome::recall. No regressions across other 2694 lib
tests. Total genome:: tests now ~79 (genome stack PRs 1-5 + this).

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack (working-
  set-manager). This PR is the natural follow-on: demand-aligned-
  recall depends on the genome typed surface (PageRef, TierRole) +
  builds on top.
- THIS PR — PR-1: pure types
- NEXT — PR-2: DemandAlignedRecall trait + CapabilityQuery +
  PersonaContext + RankedPool + RecallScoreWeights
- THEN — PR-3: LocalDemandAlignedRecall impl + scoring function +
  working-set-manager integration via #1362's bus hook

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply merged commit f5d65ee into canary May 17, 2026
4 checks passed
@joelteply joelteply deleted the feat/demand-aligned-recall-types-pr1 branch May 17, 2026 00:55
joelteply added a commit that referenced this pull request May 17, 2026
…+ composite types (#1367)

PR-2 of demand-aligned-recall (GENOME-FOUNDRY-SENTINEL Part 7).
Trait + composite types on top of PR-1's typed primitives (#1366).

What lands

- DemandAlignedRecall trait: recall + replay method signatures,
  Send + Sync + async_trait, object-safe for Arc<dyn …> dispatch.
- Typed reference newtypes: LoRALayerRef / MoEExpertRef / EngramRef
  + ArtifactRef adjacently-tagged enum for must_include hard pins.
  Adjacent tagging (kind+ref pair) because inner refs are
  #[serde(transparent)] — internally-tagged can't tag bare strings.
- CapabilityQuery — recall input.
- RecallContext — recall context with cold_start convenience.
- RankedPool — three sub-pools each with (Ref, RecallScore,
  ResidencyHint) tuples.
- RecallScoreWeights with sum-to-1.0 invariant enforced by new();
  negative weights rejected; Default matches spec baseline.
- DomainHint, RecallBudget, OutcomeWindow, TrajectoryHint,
  CompositionRef, CompositionHint, RecallTrace — supporting +
  stub-placeholder types.

Naming refinements vs raw spec

- Spec uses ResourceBudget and PersonaContext. Both clash with
  pre-existing types (comms::ResourceBudget + persona::
  cognition_io::PersonaContext + shared/generated/recipe). The TS
  master barrel re-exports the whole genome module and the duplicate
  symbols broke compilation.
- Renamed to RecallBudget and RecallContext. More precise anyway —
  they are specific to recall, not generic system-wide.

Tests

13 new test functions (28 total including ts-rs export_bindings).
trait_is_object_safe pins Arc<dyn …> dispatch; everything else
pins serde wire shapes + the sum-to-1.0 weights invariant. 28/28
pass. No regressions across other 2730 lib tests.

Clippy baseline bump 148→154 — drift from canary HEAD; the +6
warnings are NOT from genome code (zero clippy hits in
genome/recall_trait). Same pattern as PR-1 (#1346 bumped 146→148).
Bumping here so the ratchet stays meaningful for the next PR.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- THIS PR — DAR PR-2: trait + composite types
- NEXT — DAR PR-3: scoring + impl + working-set integration

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request May 17, 2026
…ctor curves (#1371)

PR-3a of demand-aligned-recall (GENOME-FOUNDRY-SENTINEL Part 7).
Pure math, no I/O, no async. Combines the five scoring factors into
a RecallScore + ships the per-factor curves the spec names.

Stacked on PR-2 (#1367 + #1370 codex hotfix). PR-3b will ship
LocalDemandAlignedRecall impl that composes recall_scoring with the
working-set-manager (#1362's bus hook) + the genome catalog.

What lands

- grid_penalty(latency_ms) -> f32 — federation peer cost curve.
  0.6 * exp(-latency_ms / 100.0). Tuned to hit the spec's four
  reference points (same-LAN ~0.55, same-region ~0.36, cross-
  region ~0.08, slow near 0).
- recency_decay(last_used_ms, now_ms, half_life_ms) -> f32 —
  exponential decay over time-since-last-use. Half-life semantics:
  exact 0.5 at one half-life. Handles clock-backward (returns 1.0)
  + zero half-life (returns 0.0 when any time has passed).
- local_role_score(role) -> f32 — spec values: Fast/Warm=1.0,
  Bench=0.6, Cold=0.3, Frozen=0.1.
- tier_proximity_for(&ResidencyHint) -> f32 — dispatches by hint
  variant: Hot→1.0, Local→local_role_score, GridPeer→grid_penalty,
  NotResident→0.0.
- score(...) — combines the five factors with RecallScoreWeights.
  Caller passes pre-computed semantic + outcome_history +
  provenance_trust (their computation belongs in PR-3b's embedding
  /sentinel/trust integrations). Returns populated RecallScore.

Design choices

- Pure function score() instead of an ArtifactCandidate struct.
  PR-3a stays math-only; candidate-aggregation lives in PR-3b.
- Combined score NOT clamped to [0,1] — out-of-range factors
  surface in the combined for debugging visibility (Joel's "never
  swallow errors" rule).
- DEFAULT_RECENCY_HALF_LIFE_MS = 24h from spec; governor tunes.

Tests

16 new tests pin every curve to its spec anchor + every edge case:

grid_penalty: matches_spec_reference_points (4 anchors) +
caps_at_0_6_for_zero_latency + monotonically_decreasing + bounded
zero to 0.6 (no negative, no NaN).

recency_decay: at_half_life_is_one_half + halves_at_each_half_life
_interval + handles_backward_clock + handles_zero_half_life +
bounded zero to one.

local_role_score: matches_spec_values + non_increasing_down_hierarchy.

tier_proximity_for: dispatches_by_residency_variant (all four hints).

score: populates_recall_score_with_computed_factors +
all_factors_one_with_default_weights_gives_one +
is_deterministic_across_calls + not_resident_can_still_score_via
_other_factors.

16/16 pass. No regressions across other 2764 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- THIS PR — DAR PR-3a: scoring function + curves (pure math)
- NEXT — DAR PR-3b: LocalDemandAlignedRecall impl + working-set
  integration

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request May 17, 2026
…ranking engine (#1372)

PR-3b of demand-aligned-recall (GENOME-FOUNDRY-SENTINEL Part 7).
Composes PR-3a's scoring function with a candidate-injection API to
produce ranked RankedPools. PR-3c adds the working-set walker that
sources candidates from the substrate; PR-3b stays pure ranking.

What lands

- CandidateArtifact — caller-provided candidate ready for scoring.
  Carries per-factor inputs (semantic, outcome, provenance) +
  residency + last-used timestamp.
- LocalDemandAlignedRecall { weights, half_life_ms } — the ranking
  engine. Thread-safe through immutability.
- new() / with_config(weights, half_life_ms) constructors.
- rank(now_ms, candidates) — pure-function ranking: scores each via
  PR-3a's score(), partitions by PageKind into layers/experts/
  engrams, sorts each sub-pool descending by RecallScore.combined,
  returns populated RankedPool.
- weights() + half_life_ms() inspectors.

Design choices

- now_ms passed in (not SystemTime::now). Replay determinism is
  mandatory per spec; reading now() would break RecallTrace replay.
- KVCache candidates silently dropped — spec's RankedPool has three
  sub-pools (layers/experts/engrams); KV cache is working-set state.
- NaN-safe sort via partial_cmp + Ordering::Equal fallback.
- trace_ref = Uuid::from_u128(now_ms) — deterministic placeholder;
  PR-3c replaces with richer RecallTrace.

What is deliberately deferred (PR-3c)

- DemandAlignedRecall trait impl (needs working-set + genome
  catalog sourcing)
- Federation sourcing (RecallScope::Federation / LocalThenGrid)
- RecallTrace replay backing store (separate sentinel PR)
- Embedding model integration

Tests

13 new tests pin the ranking behavior:
- new + with_config preserve config
- rank empty → empty pools (no error)
- rank partitions by PageKind correctly
- rank sorts each sub-pool descending by combined
- KVCache silently dropped
- score factors round-trip from PR-3a's score()
- rank is deterministic across calls (replay)
- NotResident still scored at lower combined (sentinel surface)
- Tier ordering when other factors equal (Fast > Bench > Cold >
  Frozen)
- composition_hint placeholder + trace_ref determinism pinned

13/13 pass. No regressions across other 2788 lib tests.

Clippy baseline bump 154→156 — drift from recent canary merges
(zero clippy hits in genome/recall_impl other than the doc-list
warnings I just fixed). Same pattern as PR-1 (146→148) and PR-2
(148→154).

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- THIS PR — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- NEXT — DAR PR-3c: working-set walker + trait impl + Runtime
  wiring

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request May 17, 2026
…rce seam (#1374)

PR-3c of demand-aligned-recall. Wires `DemandAlignedRecall` trait
impl on `LocalDemandAlignedRecall` + introduces `CandidateSource`
trait as the seam between the ranking engine and the substrate
candidate sources. PR-3d will wrap the working-set-manager
(#1362's bus hook) as a CandidateSource impl; PR-3c stays
substrate-agnostic.

Why this split

PR-3c locks the source seam first. PR-3d adds the working-set
walker as one impl; future PRs add the genome catalog walker +
federation peer source. Each is independently testable.

What lands

- CandidateSource trait — async fn fetch(query, context) ->
  Vec<CandidateArtifact>. Send + Sync + async_trait for tokio.
  Object-safe; PR-3d's working-set walker is one impl.
- LocalDemandAlignedRecall.source: Option<Arc<dyn CandidateSource>>
  — optional injection. None = empty-pool mode (legitimate "no
  candidates locally; try federation" signal). Some = trait
  impl's recall() dispatches to source.fetch() then rank().
- with_source(source) constructor.
- with_config_and_source(weights, half_life, source) constructor
  for governor-driven config + source wiring.
- DemandAlignedRecall trait impl on LocalDemandAlignedRecall:
  - recall(query, context) — fetches via source, scores via rank()
    with SystemTime::now() (rank() stays pure with explicit
    now_ms threading for replay determinism)
  - replay(trace) — returns typed RecallError::ScopeUnreachable
    with "RecallTraceStore (sentinel PR); not yet implemented in
    PR-3c". Per never-swallow-errors: typed refusal beats silent
    empty pool. When sentinel ships RecallTraceStore, this test
    flips to expect Ok(pool).

Design choices

- Source is Option, not required. The no-source path returns
  empty — useful for unit tests that don't need substrate +
  diagnostic tooling that wants a recall engine without
  candidate plumbing.
- `recall()` reads SystemTime::now at the trait entry. The
  internal rank() still takes explicit now_ms; replay
  determinism preserved at the pure layer, live recall at the
  trait layer. This is the cleanest decoupling I could find that
  satisfies both spec asks.
- PR-3c scope: no scope filtering, no freshness enforcement, no
  budget filtering. The CandidateSource does query-aware pruning
  in its fetch(); PR-3d's working-set walker filters by
  RecallScope::Local. Future PRs add the rest.

Tests

5 new tests on the PR-3c surface:
- recall_dispatches_through_dyn_demand_aligned_recall — Arc<dyn>
  object-safety
- recall_without_source_returns_empty_pool_not_error — empty-pool
  contract
- recall_with_source_dispatches_to_fetch_and_ranks — fetch call
  count + candidate-in-pool round-trip
- with_config_and_source_preserves_all_three
- replay_returns_typed_not_implemented_refusal_in_pr3c — pins the
  typed refusal so sentinel PR has a regression check to flip

18/18 pass on genome::recall_impl (13 PR-3b + 5 PR-3c). No
regressions across other 2802 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- THIS PR — DAR PR-3c: trait impl + CandidateSource seam
- NEXT — DAR PR-3d: WorkingSetCandidateSource wrapping #1362's
  bus hook + concrete walker for the persona's working set

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request May 17, 2026
…#1378)

The architectural payoff of the genome stack lands here. A persona's
page_in calls populate the working set (#1355); this source reads
that same working set to surface "what's already hot" candidates
that LocalDemandAlignedRecall (#1372 + #1374) ranks via the scoring
function (#1371).

End-to-end loop closed:
  page_in(persona, page) → WorkingSet.pages updated → bus publishes
  PageFault (#1362) → recall(query, ctx) → working_set_snapshot →
  CandidateArtifact per resident page → rank() → RankedPool

What lands

- WorkingSetCandidateSource struct holding
  Arc<LocalWorkingSetManager>
- CandidateSource::fetch impl that:
  - reads persona's working_set_snapshot
  - returns empty Vec on unregistered persona (no error — cold-
    start signal callers may try federation)
  - translates each ResidentPage → CandidateArtifact with
    ResidencyHint::Hot { role } (resident = hot by definition)
  - preserves PageKind for downstream sub-pool partitioning
  - sets NEUTRAL_FACTOR_STUB (0.5) for semantic / outcome_history
    / provenance_trust factors (dedicated integrations land in
    separate PRs)
- NEUTRAL_FACTOR_STUB public constant for the contract

Design choices

- Snapshot the working set via the manager's working_set_snapshot
  helper (cloned) rather than holding the RwLock across the fetch
  await. Same pattern as #1362's bus_arc hook.
- Object-safe: works through Arc<dyn CandidateSource> per PR-3c's
  contract.
- All resident pages map to Hot residency. PR-3e (or a separate
  catalog walker PR) will add Local{role=Bench/Cold/Frozen} for
  candidates outside the working set but resident in the genome
  catalog.
- Stub-0.5 factors documented inline + via NEUTRAL_FACTOR_STUB
  constant. When the embedding / sentinel / trust integrations
  land, they replace the stubs without re-touching this file.

What is deliberately deferred

- Genome catalog walker (Bench/Cold/Frozen tier sources) — needs
  the catalog module
- Federation peer source — needs federation registry
- Embedding integration (semantic factor) — separate Lane H slice
- Sentinel outcome lookup (outcome_history factor) — sentinel PR
- Trust registry lookup (provenance_trust factor) — separate PR

Tests

7 new tests, all end-to-end with real LocalWorkingSetManager +
page_in calls:
- fetch_unregistered_persona_returns_empty_not_error
- fetch_registered_empty_working_set_returns_empty
- fetch_after_page_in_returns_resident_pages_as_hot_candidates —
  the payoff test
- translation_preserves_page_kind_for_sub_pool_partitioning —
  layer → layers, expert → experts, engram → engrams
- translation_uses_neutral_factor_stubs_for_non_tier_factors —
  pins the contract so embedding-integration PRs flip it
- source_is_object_safe_for_arc_dyn_dispatch — through PR-3c's
  Arc<dyn CandidateSource>
- end_to_end_page_in_then_recall_returns_ranked_pool — full
  pipeline: page_in → WorkingSetCandidateSource ::fetch →
  LocalDemandAlignedRecall::recall → RankedPool with the
  paged-in artifacts ranked correctly

7/7 pass. No regressions across other 2822 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- THIS PR — DAR PR-3d: WorkingSetCandidateSource (the payoff)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request May 17, 2026
…1380)

Combines multiple CandidateSource impls into one, with optional
deduplication by artifact id. Sets up the extensibility seam so
future PRs (genome catalog walker, federation peer source,
must-include resolver) add sources without re-wiring
LocalDemandAlignedRecall.

What lands

- CompositeCandidateSource { sources, dedup }
- DedupPolicy::None — return all candidates from all sources (a
  single artifact may appear N times if N sources surface it).
  Useful for audit-trail callers.
- DedupPolicy::ByArtifactId — keep first occurrence per (kind,
  artifact_id) tuple in source-iteration order. Most callers want
  this (prevents double-counting a resident page that also
  surfaces via federation lookup).
- CandidateSource::fetch impl: fans out to all sources
  concurrently via futures::future::join_all, merges, dedups.
- new(sources, dedup) + with_default_dedup(sources) constructors.
- source_count() + dedup_policy() inspector methods.

Design choices

- futures::future::join_all for fan-out (concurrent, unbounded).
  Acceptable for ≤5 sources currently; federation peer counts may
  need bounding later — when that happens, this fn changes
  internals without breaking the trait.
- Dedup is configurable per composite. Most production wiring
  uses ByArtifactId; replay traces may use None for audit fidelity.
- Different PageKind with same artifact_id treated as distinct
  candidates (a layer-page reference and an engram-page reference
  happen to share the underlying artifact id; recall keeps them
  separate so the sub-pool partitioning is correct).
- Composite itself is object-safe — composites of composites
  valid for future hierarchical wiring.

What is deliberately deferred

- Source priority ordering — first-hit-wins per dedup. A future
  PR may add weighted merging.
- Per-source error isolation — fetch returns Vec, not Result. The
  underlying trait method also returns Vec; widening the trait
  would be a separate concern.
- Bounded concurrent fan-out — join_all is unbounded. Fine for
  the current source count; needs revisit when federation peers
  scale.

Tests

9 new tests pin the composite's behaviors:
- empty_composite_returns_empty_vec — no-error empty contract
- single_source_composite_passes_through — degenerate case
- fan_out_invokes_every_source_exactly_once — per-call accounting
- merge_preserves_source_iteration_order — dedup correctness
  depends on this
- dedup_none_preserves_all_duplicates
- dedup_by_artifact_id_keeps_first_occurrence_only
- dedup_treats_different_page_kinds_as_distinct
- with_default_dedup_uses_by_artifact_id
- composite_is_object_safe_as_dyn_candidate_source

9/9 pass. No regressions across other 2834 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource
- THIS PR — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- NEXT — DAR PR-3f or later: catalog walker + federation source +
  must-include resolver, all composing through this PR's seam

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request May 17, 2026
#1382)

Resolves CapabilityQuery.must_include hard pins as candidates per
GENOME-FOUNDRY-SENTINEL Part 7: "Hard pins — recall MUST include
these in the RankedPool even if their score is low. Used for
persona-private LoRA layers and sticky engrams."

Plays through the composite seam shipped in PR-3e: wired AFTER a
resident source like WorkingSetCandidateSource with ByArtifactId
dedup, must-include items that ARE resident get the resident
source's Hot residency + factor data; must-include items NOT
resident get this source's NotResident placeholder (still ranked,
just lower combined score).

What lands

- MustIncludeCandidateSource — zero-state unit struct (no Arc state
  needed; the source is pure-function over the query)
- CandidateSource::fetch impl that:
  - reads query.must_include Vec<ArtifactRef>
  - maps each variant (LoRALayer / MoEExpert / Engram) to a
    CandidateArtifact with the appropriate PageKind
  - marks every must-include candidate as ResidencyHint::
    NotResident { acquirable_from: SentinelRefinement }
  - uses NEUTRAL_FACTOR_STUB (0.5) for the three non-tier factors,
    same convention as WorkingSetCandidateSource (PR-3d)

Recommended composite wiring

  let composite = CompositeCandidateSource::with_default_dedup(vec![
      Arc::new(WorkingSetCandidateSource::new(mgr)),     // Hot first
      Arc::new(MustIncludeCandidateSource::new()),       // Pins
      // future: catalog walker, federation source
  ]);

Spec contract met: every hard-pinned artifact surfaces in the
RankedPool; if it's resident, it gets full residency-aware score;
if not, it still appears (at lower combined) so composition can
see "this was pinned but isn't here yet — schedule the foundry."

Tests

6 new tests:
- empty_must_include_returns_empty_candidates (no-error empty
  contract)
- variant_mapping_preserves_page_kind (LoRALayer/MoEExpert/Engram
  variants → PageKind mapping)
- must_include_marks_candidates_as_not_resident
- factors_use_neutral_stubs_consistent_with_working_set_source
- source_is_object_safe_for_dyn_dispatch
- composite_with_dedup_resident_wins_must_include_for_pinned_hot_
  artifact — the architectural payoff: resident pin keeps Hot,
  non-resident pin gets NotResident, both appear in merged Vec

6/6 pass. No regressions across other 2873 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource (working-set source)
- #1380 — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- THIS PR — DAR PR-3f: MustIncludeCandidateSource (hard-pin source)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant