feat(genome): demand-aligned-recall PR-3e — CompositeCandidateSource#1380
Merged
joelteply merged 1 commit intoMay 17, 2026
Merged
Conversation
Combines multiple CandidateSource impls into one, with optional
deduplication by artifact id. Sets up the extensibility seam so
future PRs (genome catalog walker, federation peer source,
must-include resolver) add sources without re-wiring
LocalDemandAlignedRecall.
What lands
- CompositeCandidateSource { sources, dedup }
- DedupPolicy::None — return all candidates from all sources (a
single artifact may appear N times if N sources surface it).
Useful for audit-trail callers.
- DedupPolicy::ByArtifactId — keep first occurrence per (kind,
artifact_id) tuple in source-iteration order. Most callers want
this (prevents double-counting a resident page that also
surfaces via federation lookup).
- CandidateSource::fetch impl: fans out to all sources
concurrently via futures::future::join_all, merges, dedups.
- new(sources, dedup) + with_default_dedup(sources) constructors.
- source_count() + dedup_policy() inspector methods.
Design choices
- futures::future::join_all for fan-out (concurrent, unbounded).
Acceptable for ≤5 sources currently; federation peer counts may
need bounding later — when that happens, this fn changes
internals without breaking the trait.
- Dedup is configurable per composite. Most production wiring
uses ByArtifactId; replay traces may use None for audit fidelity.
- Different PageKind with same artifact_id treated as distinct
candidates (a layer-page reference and an engram-page reference
happen to share the underlying artifact id; recall keeps them
separate so the sub-pool partitioning is correct).
- Composite itself is object-safe — composites of composites
valid for future hierarchical wiring.
What is deliberately deferred
- Source priority ordering — first-hit-wins per dedup. A future
PR may add weighted merging.
- Per-source error isolation — fetch returns Vec, not Result. The
underlying trait method also returns Vec; widening the trait
would be a separate concern.
- Bounded concurrent fan-out — join_all is unbounded. Fine for
the current source count; needs revisit when federation peers
scale.
Tests
9 new tests pin the composite's behaviors:
- empty_composite_returns_empty_vec — no-error empty contract
- single_source_composite_passes_through — degenerate case
- fan_out_invokes_every_source_exactly_once — per-call accounting
- merge_preserves_source_iteration_order — dedup correctness
depends on this
- dedup_none_preserves_all_duplicates
- dedup_by_artifact_id_keeps_first_occurrence_only
- dedup_treats_different_page_kinds_as_distinct
- with_default_dedup_uses_by_artifact_id
- composite_is_object_safe_as_dyn_candidate_source
9/9 pass. No regressions across other 2834 lib tests.
Stack
- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource
- THIS PR — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- NEXT — DAR PR-3f or later: catalog walker + federation source +
must-include resolver, all composing through this PR's seam
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
joelteply
added a commit
that referenced
this pull request
May 17, 2026
#1382) Resolves CapabilityQuery.must_include hard pins as candidates per GENOME-FOUNDRY-SENTINEL Part 7: "Hard pins — recall MUST include these in the RankedPool even if their score is low. Used for persona-private LoRA layers and sticky engrams." Plays through the composite seam shipped in PR-3e: wired AFTER a resident source like WorkingSetCandidateSource with ByArtifactId dedup, must-include items that ARE resident get the resident source's Hot residency + factor data; must-include items NOT resident get this source's NotResident placeholder (still ranked, just lower combined score). What lands - MustIncludeCandidateSource — zero-state unit struct (no Arc state needed; the source is pure-function over the query) - CandidateSource::fetch impl that: - reads query.must_include Vec<ArtifactRef> - maps each variant (LoRALayer / MoEExpert / Engram) to a CandidateArtifact with the appropriate PageKind - marks every must-include candidate as ResidencyHint:: NotResident { acquirable_from: SentinelRefinement } - uses NEUTRAL_FACTOR_STUB (0.5) for the three non-tier factors, same convention as WorkingSetCandidateSource (PR-3d) Recommended composite wiring let composite = CompositeCandidateSource::with_default_dedup(vec![ Arc::new(WorkingSetCandidateSource::new(mgr)), // Hot first Arc::new(MustIncludeCandidateSource::new()), // Pins // future: catalog walker, federation source ]); Spec contract met: every hard-pinned artifact surfaces in the RankedPool; if it's resident, it gets full residency-aware score; if not, it still appears (at lower combined) so composition can see "this was pinned but isn't here yet — schedule the foundry." Tests 6 new tests: - empty_must_include_returns_empty_candidates (no-error empty contract) - variant_mapping_preserves_page_kind (LoRALayer/MoEExpert/Engram variants → PageKind mapping) - must_include_marks_candidates_as_not_resident - factors_use_neutral_stubs_consistent_with_working_set_source - source_is_object_safe_for_dyn_dispatch - composite_with_dedup_resident_wins_must_include_for_pinned_hot_ artifact — the architectural payoff: resident pin keeps Hot, non-resident pin gets NotResident, both appear in merged Vec 6/6 pass. No regressions across other 2873 lib tests. Stack - #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager - #1366 — DAR PR-1: pure types - #1367 + #1370 — DAR PR-2: trait + composite types - #1371 — DAR PR-3a: scoring function + per-factor curves - #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine - #1374 — DAR PR-3c: trait impl + CandidateSource seam - #1378 — DAR PR-3d: WorkingSetCandidateSource (working-set source) - #1380 — DAR PR-3e: CompositeCandidateSource (extensibility seam) - THIS PR — DAR PR-3f: MustIncludeCandidateSource (hard-pin source) Co-authored-by: Test <test@test.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR-3e of demand-aligned-recall. Combines multiple
CandidateSourceimpls into one with optional deduplication. Sets up the extensibility seam so future PRs (genome catalog walker, federation peer source, must-include resolver) add sources without re-wiringLocalDemandAlignedRecall.What lands
CompositeCandidateSource { sources, dedup }DedupPolicy::None— return all candidates from all sources (audit-trail callers).DedupPolicy::ByArtifactId— keep first occurrence per(kind, artifact_id)tuple in source-iteration order (most production callers).CandidateSource::fetchimpl: fans out viafutures::future::join_all, merges, applies dedup.new(sources, dedup)+with_default_dedup(sources)constructors.Design choices
futures::future::join_allfor concurrent fan-out. Acceptable for ≤5 sources; federation peer counts may need bounding later — that change is internal to this fn.PageKindwith sameartifact_idtreated as distinct candidates (a layer-page and an engram-page that share the artifact id stay separate so sub-pool partitioning is correct).What is deliberately deferred
fetchreturnsVec, notResult)Test plan
cargo test --lib --features metal,accelerate genome::recall_source_composite— 9/9 pass:empty_composite_returns_empty_vecsingle_source_composite_passes_throughfan_out_invokes_every_source_exactly_oncemerge_preserves_source_iteration_orderdedup_none_preserves_all_duplicatesdedup_by_artifact_id_keeps_first_occurrence_onlydedup_treats_different_page_kinds_as_distinctwith_default_dedup_uses_by_artifact_idcomposite_is_object_safe_as_dyn_candidate_sourceStack
LocalDemandAlignedRecallranking engineWorkingSetCandidateSourceCompositeCandidateSource(extensibility seam)🤖 Generated with Claude Code