Skip to content

feat(spectrum): cap-bounded forward mem_search (gap A, min_intv-bounded SMEM) - #28

Merged
nh13 merged 1 commit into
mainfrom
feat/v0.3-mem-capped
Jun 12, 2026
Merged

feat(spectrum): cap-bounded forward mem_search (gap A, min_intv-bounded SMEM)#28
nh13 merged 1 commit into
mainfrom
feat/v0.3-mem-capped

Conversation

@nh13

@nh13 nh13 commented Jun 11, 2026

Copy link
Copy Markdown

v0.3 perf series (authored fresh on main; built on #26's forward_maximal_len → (match_len, ip) and reusing #27's in-interval seed derivation).

What

Adds mem_search_capped + the prmi_mem_search_capped FFI — BWA-MEME's min_intv-bounded SMEM interval counting (gap A).

  • match_len is always exact.
  • When the true occ <= max_intv, the SA interval (sa_start, occ) is byte-identical to mem_search.
  • When the true occ > max_intv, the outward interval scan stops early and returns occ == max_intv + 1 with the wide-interval gallop skipped (the dominant cost on the repetitive tail), so sa_start is a partial bound — the caller treats occ > max_intv as "too repetitive".

How

Bracketed O(log) launch (forward_maximal_len), in-interval seed derived from the insertion point with one shares_prefix(ip-1) probe (same as mem_search_backward_truncated_span_rc), then a capped outward scan.

Tests

  • mem_search_capped_equals_uncapped_under_cap (proptest): byte-identity vs mem_search for occ <= cap, short-circuit occ > cap otherwise — across query lengths and caps straddling the true occ (incl. 0 and a huge value).
  • mem_search_capped_ffi_matches_uncapped_under_cap: the FFI surface.

Notes

FFI null-checks all three out-ptrs unconditionally (it always writes the interval, no WANT_INTERVAL flag) and uses packed_pac_bytes (the narrowing guard), not the original perf branch's raw div_ceil cast. This is the next building block toward the interval-returning truncated reseed.

Self-reviewed with /coderabbitai-review before opening (0 actionable; applied 1 nitpick — added cap=0 to the proptest spread).

Summary by CodeRabbit

  • New Features

    • Added a cap-bounded maximal forward search that limits reported occurrences and short-circuits when the cap is reached.
  • Tests

    • Added FFI regression test ensuring capped search matches uncapped results when under the cap.
    • Added property-based tests asserting capped vs. uncapped behavior and exact match-length preservation across random indices and queries.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 31fb236c-1145-4344-a209-9448309a8378

📥 Commits

Reviewing files that changed from the base of the PR and between 778340e and 82fd7e5.

📒 Files selected for processing (3)
  • prmi-sys/src/lib.rs
  • prmi-sys/tests/spectrum_ffi.rs
  • prmi/src/index/spectrum.rs
🚧 Files skipped from review as they are similar to previous changes (3)
  • prmi-sys/tests/spectrum_ffi.rs
  • prmi-sys/src/lib.rs
  • prmi/src/index/spectrum.rs

📝 Walkthrough

Walkthrough

Adds a cap-bounded forward maximal match: LearnedIndex::mem_search_capped (core), prmi_mem_search_capped (FFI) with pointer/packing/panic-safe validation, and tests ensuring byte-equality to uncapped results when not truncated and exact short-circuit behavior when truncated.

Changes

Capped forward maximal match query

Layer / File(s) Summary
Core implementation and validation
prmi/src/index/spectrum.rs
Adds LearnedIndex::mem_search_capped that validates packed PAC, computes exact match_len using forward_maximal_len, selects a seed from the insertion point, and expands the SA interval while bounding by max_intv; returns occ = max_intv + 1 when the true interval exceeds the cap while keeping match_len exact.
Core property test
prmi/src/index/spectrum.rs
Adds proptest mem_search_capped_equals_uncapped_under_cap asserting match_len equality to uncapped mem_search for all caps, equality of (sa_start, occ) when full.occ <= cap, and exact short-circuit occ == cap + 1 when full.occ > cap.
FFI entrypoint and error handling
prmi-sys/src/lib.rs
Adds prmi_mem_search_capped extern C wrapper with null/out-pointer checks, query_len validation, packed_pac_bytes/PacEncoding::Packed construction, catch_unwind around the core call, guaranteed writes to all output pointers on success, and return codes: 0 success, -1 null pointers, -2 invalid packed PAC or query_len, -3 internal panic.
FFI test and import updates
prmi-sys/tests/spectrum_ffi.rs
Rewraps use prmi_sys imports and adds mem_search_capped_ffi_matches_uncapped_under_cap test comparing prmi_mem_search_capped to prmi_mem_search, checking identity when cap ≥ occ and validating cap-1 short-circuit (occ == cap + 1, match_len unchanged).

Sequence Diagram

sequenceDiagram
  participant C as Caller
  participant FFI as prmi_mem_search_capped
  participant Core as LearnedIndex::mem_search_capped
  participant MaxLen as forward_maximal_len
  participant Expand as IntervalExpansion

  C->>FFI: call(query, cap, pac, out ptrs)
  FFI->>FFI: clear_last_error
  FFI->>FFI: validate pointers and query_len
  FFI->>FFI: pack PAC bytes
  FFI->>Core: call mem_search_capped(packed pac, query, max_intv)
  Core->>MaxLen: compute match_len and insertion point
  MaxLen-->>Core: return match_len and ip
  Core->>Expand: select seed (ip or ip-1)
  Expand->>Expand: expand left/right while occ <= max_intv
  alt interval within cap
    Expand-->>Core: return sa_start and occ
  else interval exceeds cap
    Expand-->>Core: return partial sa_start and occ = max_intv + 1
  end
  Core-->>FFI: MemMatch result
  FFI->>FFI: write out_match_len, out_sa_start, out_occ
  FFI-->>C: return 0 (success)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • fg-labs/prmi#26: Both PRs change prmi/src/index/spectrum.rs around forward_maximal_len/interval recovery; #26 refactors forward_maximal_len to return an insertion point used by this PR.
  • fg-labs/prmi#23: Shares the same forward-maximal machinery used as a foundation for the capped search implementation.
  • fg-labs/prmi#17: Related FFI search conventions and prior prmi_mem_search FFI behavior that this capped variant mirrors and tests against.

Poem

🐰 Hopping through suffix rows with care,
I count occurrences but always spare.
One exact match length, intervals capped tight,
From core to FFI the search stays right.
Tests nod—short-circuited, snug and light.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly reflects the main change: adding a cap-bounded forward mem_search function (with references to gap A and SMEM interval counting) across the codebase.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@prmi-sys/tests/spectrum_ffi.rs`:
- Around line 1307-1326: Replace the weak inequality check with a strict
equality to enforce the capped-occ contract: when calling prmi_mem_search_capped
with cap = occ - 1, assert that the returned c2occ equals cap + 1 (which is occ)
instead of only checking c2occ > cap; update the assertion on c2occ (around the
prmi_mem_search_capped call) to assert_eq!(c2occ, occ, "occ==cap+1: expected
short-circuit return of cap+1") so the test fails if the implementation returns
the full true interval count.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 41b7ff22-f407-4a31-b978-229514481afb

📥 Commits

Reviewing files that changed from the base of the PR and between c67b2e3 and 778340e.

📒 Files selected for processing (3)
  • prmi-sys/src/lib.rs
  • prmi-sys/tests/spectrum_ffi.rs
  • prmi/src/index/spectrum.rs

Comment thread prmi-sys/tests/spectrum_ffi.rs
…ed SMEM)

Adds mem_search_capped + the prmi_mem_search_capped FFI — BWA-MEME's
min_intv-bounded SMEM interval counting. The maximal match_len is exact; the SA
interval is BYTE-IDENTICAL to mem_search when the true occ <= max_intv, else the
outward interval scan stops early at occ == max_intv + 1 with the wide-interval
gallop skipped (the dominant cost on the repetitive tail), so sa_start is then a
partial bound and the caller treats occ > max_intv as too repetitive.

Bracketed O(log) launch; derives the in-interval seed from forward_maximal_len's
insertion point with one shares_prefix probe (same as the truncated span). FFI
uses packed_pac_bytes (the narrowing guard), not a raw div_ceil cast.
Byte-identity proptest (capped == mem_search for occ <= cap) + an FFI test. The
next building block toward the interval-returning truncated reseed. v0.3 perf
series.
@nh13
nh13 force-pushed the feat/v0.3-mem-capped branch from 778340e to 82fd7e5 Compare June 11, 2026 22:13
@nh13

nh13 commented Jun 11, 2026

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 merged commit 832a1fb into main Jun 12, 2026
4 checks passed
@nh13
nh13 deleted the feat/v0.3-mem-capped branch June 12, 2026 06:02
nh13 added a commit that referenced this pull request Jun 12, 2026
Adds mem_search_backward_truncated_interval + the
prmi_mem_search_backward_truncated_interval FFI — the consumer's
zz_left_span_reseed seed emission in one call: the largest left span L with
occ(L) >= min_intv (floored at anchor_len), returned with its EXACT forward SA
interval (MemMatch { match_len: L, sa_start, occ }).

Locates L by a gallop-down + binary search whose predicate is
mem_search_capped(window, min_intv).occ >= min_intv (#28) — exact, never gallops
past min_intv — then recovers L's interval with one uncapped mem_search. span_max
from mem_search_backward (or, when est_hint != 0, the .isa-hinted
mem_search_backward_from_hint). est_hint only affects probe count.

Byte-identical to an independent oracle (truncated_span_oracle's L* + mem_search)
across the proptest; +FFI plumbing test. Checked anchor_end, packed-pac guard,
packed_pac_bytes in the FFI. This is the slow-but-exact binary-search form; the
RC-downward _interval_rc swaps into the FFI later. v0.3 perf series.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant