Skip to content

feat(spectrum): min_intv-truncated backward reseed interval + FFI - #29

Merged
nh13 merged 1 commit into
mainfrom
feat/v0.3-trunc-interval
Jun 12, 2026
Merged

feat(spectrum): min_intv-truncated backward reseed interval + FFI#29
nh13 merged 1 commit into
mainfrom
feat/v0.3-trunc-interval

Conversation

@nh13

@nh13 nh13 commented Jun 12, 2026

Copy link
Copy Markdown

v0.3 perf series (authored fresh on main; built on #28's mem_search_capped).

What

Adds mem_search_backward_truncated_interval + the prmi_mem_search_backward_truncated_interval FFI — the consumer's zz_left_span_reseed seed emission in one call: the largest left-extension span L with occ(L) >= min_intv (floored at anchor_len), returned with its exact forward SA interval (match_len: L, sa_start, occ).

How

Gallop-down + binary search to find L, predicate mem_search_capped(window, min_intv).occ >= min_intv (#28 — exact, never gallops past min_intv), then one uncapped mem_search to recover L's interval. The binary search is sound because the predicate is monotone (occ(L) non-increasing in L). span_max from mem_search_backward (or, with a valid est_hint, the hinted mem_search_backward_from_hint).

Byte-identity

Verified against an independent oracle (truncated_span_oracle's L* + mem_search) across the proptest; + an FFI plumbing/contract test.

Self-review caught a real doc bug

/coderabbitai-review (pre-PR) added an est_hint != 0 test asserting "only affects probe count" — it failed, revealing that mem_search_backward_from_hint trusts the hint (it must be the inverse-SA of the anchor's locus, not any in-interval index). Corrected the docs to state the no_search contract and removed the test that used an invalid hint (the est_hint path is covered by #19's reference-lifted mem_search_backward_from_hint tests; a random-read proptest can't construct a valid locus hint).

Scope

This is the slow-but-exact binary-search form; the faster RC-downward _interval_rc swaps into the FFI later. Checked anchor_end, packed-pac guard, packed_pac_bytes in the FFI.

Summary by CodeRabbit

  • New Features

    • Added backward search functionality with truncated interval support to optimize sequence searching with occurrence count thresholds.
  • Tests

    • Added comprehensive test coverage for the new backward search capability, including validation of occurrence count extremes and interval truncation behavior.

Adds mem_search_backward_truncated_interval + the
prmi_mem_search_backward_truncated_interval FFI — the consumer's
zz_left_span_reseed seed emission in one call: the largest left span L with
occ(L) >= min_intv (floored at anchor_len), returned with its EXACT forward SA
interval (MemMatch { match_len: L, sa_start, occ }).

Locates L by a gallop-down + binary search whose predicate is
mem_search_capped(window, min_intv).occ >= min_intv (#28) — exact, never gallops
past min_intv — then recovers L's interval with one uncapped mem_search. span_max
from mem_search_backward (or, when est_hint != 0, the .isa-hinted
mem_search_backward_from_hint). est_hint only affects probe count.

Byte-identical to an independent oracle (truncated_span_oracle's L* + mem_search)
across the proptest; +FFI plumbing test. Checked anchor_end, packed-pac guard,
packed_pac_bytes in the FFI. This is the slow-but-exact binary-search form; the
RC-downward _interval_rc swaps into the FFI later. v0.3 perf series.
@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8190e41e-1e73-43b2-8a10-8a84cc238944

📥 Commits

Reviewing files that changed from the base of the PR and between 832a1fb and 2d9d225.

📒 Files selected for processing (3)
  • prmi-sys/src/lib.rs
  • prmi-sys/tests/spectrum_ffi.rs
  • prmi/src/index/spectrum.rs

📝 Walkthrough

Walkthrough

This PR introduces mem_search_backward_truncated_interval, a backward reseed primitive that finds the largest left span meeting a minimum occurrence threshold, then returns its exact forward interval. The implementation spans core Rust logic with oracle-based property testing and a paired C-ABI export with integration tests.

Changes

Backward Truncated Interval Search

Layer / File(s) Summary
Core implementation: backward truncated interval search
prmi/src/index/spectrum.rs
LearnedIndex::mem_search_backward_truncated_interval computes the maximal backward span, then uses gallop-down and binary search to find the largest length L >= anchor_len where occurrence count meets min_intv, finally recovering the exact SA interval via forward search for that length window.
Oracle-based test verification
prmi/src/index/spectrum.rs
truncated_interval_oracle brute-forces the correct truncated span L*, then returns its exact forward interval; proptest validates the implementation against the oracle across varied min_intv and pivot values.
FFI wrapper and integration tests
prmi-sys/src/lib.rs, prmi-sys/tests/spectrum_ffi.rs
prmi_mem_search_backward_truncated_interval exports the method as a C-ABI function with pointer validation, range checks, and panic-to-error translation; mem_search_backward_truncated_interval_ffi test confirms min_intv=1 matches maximal extension and huge min_intv floors to anchor values.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • fg-labs/prmi#28: The new implementation directly calls mem_search_capped(..., min_intv) introduced in this PR to find the truncation point L*.
  • fg-labs/prmi#17: This PR's mem_search_backward_truncated_interval directly builds its span_max from LearnedIndex::mem_search_backward introduced there.
  • fg-labs/prmi#27: The test oracle reuses the truncated_span_oracle helper from this PR to compute the same min_intv-truncated backward span before recovering the exact forward interval.

Poem

🐰 A backward leap through alphabet's vast sea,
Truncating spans till minimum decree,
Galloping down, then binary we go,
To find the largest L that meets the flow,
Exact intervals dance where thresholds show!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(spectrum): min_intv-truncated backward reseed interval + FFI' clearly and specifically summarizes the main changes: addition of a new backward reseed interval feature for the spectrum index with FFI bindings.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@nh13

nh13 commented Jun 12, 2026

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@nh13
nh13 merged commit adfc180 into main Jun 12, 2026
4 checks passed
@nh13
nh13 deleted the feat/v0.3-trunc-interval branch June 12, 2026 07:07
nh13 added a commit that referenced this pull request Jun 12, 2026
Add `forward_truncate_below_maximal`, the FORWARD analogue of the TRUNC_IV
backward family (#27/#29/#30) for the reseed's right bound. Given a query and its
already-found maximal match whose `occ < min_intv`, it finds the largest length
L < match_len whose prefix occurs >= min_intv times and returns that interval (or
the zero MemMatch when none qualifies).

Forward PREFIX intervals nest directly (fixed start, shrinking end: query[..L] is
a superset of query[..L+1]), unlike the LEFT reseed where prepending bases breaks
forward nesting and forces the RC walk. So the walk carries [lo, hi) outward from
the maximal interval as L shrinks — a short shares_prefix scan capped at min_intv
per step, no model launch. `want_interval == false` returns only L* (the driver's
next_pivot use); `true` recovers the exact interval by galloping find_boundary out
from the capped walk's interior seed (seed-independent, so byte-identical to a
cold mem_search).

Adapted to main: 6-arg shares_prefix/ref_less (no keyed_compare_mask). Verified
byte-identical against an independent per-length mem_search oracle across
reference-lifted queries and a spread of min_intv. Pub primitive with no in-tree
caller until the collect reseed module lands (the proptest is its only consumer).
nh13 added a commit that referenced this pull request Jun 12, 2026
Add `forward_truncate_below_maximal`, the FORWARD analogue of the TRUNC_IV
backward family (#27/#29/#30) for the reseed's right bound. Given a query and its
already-found maximal match whose `occ < min_intv`, it finds the largest length
L < match_len whose prefix occurs >= min_intv times and returns that interval (or
the zero MemMatch when none qualifies).

Forward PREFIX intervals nest directly (fixed start, shrinking end: query[..L] is
a superset of query[..L+1]), unlike the LEFT reseed where prepending bases breaks
forward nesting and forces the RC walk. So the walk carries [lo, hi) outward from
the maximal interval as L shrinks — a short shares_prefix scan capped at min_intv
per step, no model launch. `want_interval == false` returns only L* (the driver's
next_pivot use); `true` recovers the exact interval by galloping find_boundary out
from the capped walk's interior seed (seed-independent, so byte-identical to a
cold mem_search).

Adapted to main: 6-arg shares_prefix/ref_less (no keyed_compare_mask). Verified
byte-identical against an independent per-length mem_search oracle across
reference-lifted queries and a spread of min_intv. Pub primitive with no in-tree
caller until the collect reseed module lands (the proptest is its only consumer).
nh13 added a commit that referenced this pull request Jun 12, 2026
Add `forward_truncate_below_maximal`, the FORWARD analogue of the TRUNC_IV
backward family (#27/#29/#30) for the reseed's right bound. Given a query and its
already-found maximal match whose `occ < min_intv`, it finds the largest length
L < match_len whose prefix occurs >= min_intv times and returns that interval (or
the zero MemMatch when none qualifies).

Forward PREFIX intervals nest directly (fixed start, shrinking end: query[..L] is
a superset of query[..L+1]), unlike the LEFT reseed where prepending bases breaks
forward nesting and forces the RC walk. So the walk carries [lo, hi) outward from
the maximal interval as L shrinks — a short shares_prefix scan capped at min_intv
per step, no model launch. `want_interval == false` returns only L* (the driver's
next_pivot use); `true` recovers the exact interval by galloping find_boundary out
from the capped walk's interior seed (seed-independent, so byte-identical to a
cold mem_search).

Adapted to main: 6-arg shares_prefix/ref_less (no keyed_compare_mask). Verified
byte-identical against an independent per-length mem_search oracle across
reference-lifted queries and a spread of min_intv. Pub primitive with no in-tree
caller until the collect reseed module lands (the proptest is its only consumer).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant