Skip to content

feat(spectrum): forward min_intv-truncated extension primitive - #31

Merged
nh13 merged 1 commit into
mainfrom
feat/v0.3-fwd-truncate
Jun 12, 2026
Merged

feat(spectrum): forward min_intv-truncated extension primitive#31
nh13 merged 1 commit into
mainfrom
feat/v0.3-fwd-truncate

Conversation

@nh13

@nh13 nh13 commented Jun 12, 2026

Copy link
Copy Markdown

v0.3 perf series — PR-D1 of the collect_smems carve (the forward primitive the fused reseed driver needs; the collect module + FFI land in PR-D2 on top of this).

What

Adds forward_truncate_below_maximal, the forward analogue of the TRUNC_IV backward family (#27/#29/#30), for the reseed's right (forward) bound. Given a query and its already-found maximal match whose occ < min_intv, it finds the largest length L < match_len whose prefix occurs >= min_intv times and returns that interval (or the zero MemMatch when none qualifies).

Why a forward walk works here (and the backward reseed needed RC)

Forward prefix intervals nest directly — fixed start, shrinking end: query[..L] is a superset of query[..L+1] in forward SA. So the walk carries [lo, hi) outward from the maximal interval as L shrinks — a short shares_prefix scan capped at min_intv per step, no model launch. (Contrast the LEFT reseed, where prepending bases breaks forward nesting and forces _interval_rc's RC walk.) occ is monotone non-increasing in L, so the first (largest) L reaching min_intv is L*.

want_interval == false returns only L* (the driver's next_pivot use); true recovers the exact interval by galloping find_boundary out from the capped walk's interior seed — find_boundary is seed-independent, so the boundaries (and occ) are byte-identical to a cold mem_search; the tight seed only removes the model launch.

Byte-identity

Verified against an independent per-length mem_search oracle across reference-lifted queries and min_intv ∈ {1,2,3,5,20,1e6} (covering the no-truncation, crossing, and no-qualifying-L → zero paths), plus the want_interval=false span-only return.

Notes

  • Adapted to main: 6-arg shares_prefix/ref_less (no keyed_compare_mask).
  • Like the merged _span_rc, it's an internal span primitive with no packed-pac guard (the caller's prior mem_search validates the pac); the guarded siblings are the public-FFI entrypoints, which this is not.
  • pub with no in-tree caller until the collect reseed module lands (PR-D2) — the established stacked-primitive pattern; the proptest is its only current consumer.
  • Pre-PR /coderabbitai-review: 0 actionable findings.

Summary by CodeRabbit

  • New Features

    • Forward truncation can find the largest shorter prefix that still meets a minimum occurrence threshold and optionally return either just the match length or the exact interval.
  • Bug Fixes / Robustness

    • Truncation now fails closed on malformed or out-of-range inputs, returning an empty match instead of crashing.
  • Tests

    • Added brute-force oracle, property-based, and unit tests to validate truncation behavior and malformed-input handling.

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 29e3acd2-be41-4027-ad1b-9159de3fc3b9

📥 Commits

Reviewing files that changed from the base of the PR and between 7ff9c1a and f422159.

📒 Files selected for processing (1)
  • prmi/src/index/spectrum.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • prmi/src/index/spectrum.rs

📝 Walkthrough

Walkthrough

Adds LearnedIndex::forward_truncate_below_maximal: a capped outward SA-interval expansion that finds the largest shorter prefix L < maximal.match_len with occurrences >= min_intv, optionally recovering exact SA boundaries; includes a brute-force oracle, proptest validation, and a unit test for fail-closed malformed maximal inputs.

Changes

Forward Truncation Primitive

Layer / File(s) Summary
Core outward-walk truncation algorithm
prmi/src/index/spectrum.rs
forward_truncate_below_maximal validates maximal, walks outward from its SA interval while decreasing prefix length and expanding the interval only until the width meets min_intv, records the best feasible length L, and optionally recovers exact SA boundaries via find_boundary when want_interval is true.
Oracle and property/unit tests
prmi/src/index/spectrum.rs
Adds forward_truncate_oracle brute-force helper, forward_truncate_below_maximal_equals_oracle proptest comparing the new algorithm to the oracle (validating both interval and span-only behaviors), and forward_truncate_below_maximal_fails_closed_on_bad_input unit test ensuring malformed maximal and undersized pac return the zero MemMatch.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • fg-labs/prmi#17: Introduced MemMatch/mem_search APIs used by the oracle and truncation logic.

Poem

🐰 I hop the SA fields at play,
Truncating forward, finding L today;
Min cap held tight, I wander wide,
Oracle watches by my side,
Boundaries found — a tidy stride.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: adding a forward min_intv-truncated extension primitive to the spectrum module.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ef16a522-6eed-4023-854e-6db5c8fd4b3b

📥 Commits

Reviewing files that changed from the base of the PR and between c560c3c and f9b10af.

📒 Files selected for processing (1)
  • prmi/src/index/spectrum.rs

Comment thread prmi/src/index/spectrum.rs Outdated
@nh13
nh13 force-pushed the feat/v0.3-fwd-truncate branch from f9b10af to 7ff9c1a Compare June 12, 2026 20:54
@nh13

nh13 commented Jun 12, 2026

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@prmi/src/index/spectrum.rs`:
- Around line 2217-2253: forward_truncate_below_maximal must validate packed
PACs and fail closed before calling shares_prefix to avoid panics on undersized
packed decoding; add an early guard in forward_truncate_below_maximal that, when
enc == PacEncoding::Packed, checks the provided pac buffer is large enough for
l_pac (or otherwise meets the packed-decode size requirements) and returns zero
on failure, so no packed decode unwraps occur inside shares_prefix or downstream
calls.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cf6d3336-0ad0-4ab7-a375-bcf71f954317

📥 Commits

Reviewing files that changed from the base of the PR and between f9b10af and 7ff9c1a.

📒 Files selected for processing (1)
  • prmi/src/index/spectrum.rs

Comment thread prmi/src/index/spectrum.rs
Add `forward_truncate_below_maximal`, the FORWARD analogue of the TRUNC_IV
backward family (#27/#29/#30) for the reseed's right bound. Given a query and its
already-found maximal match whose `occ < min_intv`, it finds the largest length
L < match_len whose prefix occurs >= min_intv times and returns that interval (or
the zero MemMatch when none qualifies).

Forward PREFIX intervals nest directly (fixed start, shrinking end: query[..L] is
a superset of query[..L+1]), unlike the LEFT reseed where prepending bases breaks
forward nesting and forces the RC walk. So the walk carries [lo, hi) outward from
the maximal interval as L shrinks — a short shares_prefix scan capped at min_intv
per step, no model launch. `want_interval == false` returns only L* (the driver's
next_pivot use); `true` recovers the exact interval by galloping find_boundary out
from the capped walk's interior seed (seed-independent, so byte-identical to a
cold mem_search).

Adapted to main: 6-arg shares_prefix/ref_less (no keyed_compare_mask). Verified
byte-identical against an independent per-length mem_search oracle across
reference-lifted queries and a spread of min_intv. Pub primitive with no in-tree
caller until the collect reseed module lands (the proptest is its only consumer).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant