perf(spectrum): recover occurrence intervals by bounded outward walks

[nh13]

First of a three-PR stack of upstreamable prmi boundary-search / SMEM-collection perf, each byte-identical to main and gated by proptests. Suggested reading order: this PR, then perf/boundary-search, then perf/tokenize-amortize.

What

Recover a maximal match's SA occurrence interval by bounded outward walks from the insertion point instead of a full-SA gallop. occ_lower walks down and occ_upper walks up from the index the search already reached, probing each interval position once, and delegate only the tail (past a small linear cap) to the galloping find_boundary. mem_search, mem_search_warmstart, and forward_truncate_below_maximal are rewired onto these; the latter also drops its full-SA find_boundary(0, sa_num, …) gallop.

Why

Real occurrence intervals are tiny (occ p99 ≈ 4), so the linear walk resolves ~all of them in a handful of adjacent probes rather than a log2(sa_num) gallop, cutting SA probes per read.

Correctness

Byte-identical. occ_lower/occ_upper return exactly what the galloping find_boundary would (a capped linear scan of a monotone predicate, with a gallop tail), gated by occ_lower_equals_find_boundary_lower and the existing mem_search/forward_truncate_below_maximal oracle proptests.

Summary by CodeRabbit

• Performance Improvements

  • Improved interval recovery during lookups in small-interval scenarios, speeding up searches and truncation-related operations.
  • Reduced reliance on more expensive boundary scanning by using a faster bounded local recovery first, then falling back only when necessary.

• Bug Fixes

  • Preserved exact boundary/interval results while improving efficiency, ensuring endpoints remain consistent across relevant search paths.

• Tests

  • Added coverage to verify the optimized boundary recovery matches prior behavior across randomized cases.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4f597790-b537-498e-97fd-f51491bd5f49

📥 Commits

Reviewing files that changed from the base of the PR and between ad9d557 and 0213444.

📒 Files selected for processing (1)

• prmi/src/index/spectrum.rs

---

Walkthrough

Interval-boundary recovery in three LearnedIndex paths now uses bounded linear-walk helpers with fallback to find_boundary; a randomized test checks the new helpers against the original boundary oracle.

Changes

occ_lower/occ_upper boundary recovery

Layer / File(s)	Summary
occ_lower/occ_upper helpers prmi/src/index/spectrum.rs	Adds occ_lower and occ_upper, which walk within a cap before falling back to find_boundary for exact boundary recovery.
mem_search / mem_search_warmstart wiring prmi/src/index/spectrum.rs	Updates the no-kmt interval recovery paths to use occ_lower(ip, ...) and occ_upper(ip, ...) instead of galloping find_boundary probes.
forward_truncate_below_maximal wiring prmi/src/index/spectrum.rs	Updates exact interval recovery for want_interval == true to use occ_lower(lo, ...) and occ_upper(hi, ...) in place of full-SA find_boundary calls.
Equivalence test prmi/src/index/spectrum.rs	Adds occ_lower_equals_find_boundary_lower to compare the new helpers against find_boundary across randomized cases.

Estimated code review effort: 4 (Complex) | ~40 minutes

Possibly related PRs

• fg-labs/prmi#21: Touches the same boundary-search loops in prmi/src/index/spectrum.rs.
• fg-labs/prmi#26: Modifies the same no-kmt interval recovery path that this PR rewires.
• fg-labs/prmi#48: Also changes forward_truncate_below_maximal in prmi/src/index/spectrum.rs.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main performance change: bounded outward walks to recover occurrence intervals.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch perf/interval-recovery

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@prmi/src/index/spectrum.rs`:
- Around line 5106-5161: Add a direct oracle check for occ_upper in
occ_lower_equals_find_boundary_lower by comparing it against find_boundary with
the shares_prefix predicate, instead of only asserting occ_upper(ip) ==
occ_upper(linear). Keep the existing occ_lower/find_boundary comparison, and use
the same unique symbols occ_upper, find_boundary, shares_prefix, and linear/ip
variables so the test explicitly verifies occ_upper’s boundary against the
gallop-based reference.
- Around line 4187-4227: Add #[inline] to the hot-path helper methods occ_lower
and occ_upper in spectrum.rs, since they mirror the existing inlined helper
chain used by find_boundary and are called on the mem_search comparator tail.
Keep the current logic unchanged and apply the attribute directly on each
function definition so the compiler can inline these bounded-walk recovery paths
alongside find_boundary, ref_less, shares, and lcp_at.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 375231b7-97b9-440a-b8d3-6cee677d6c59

📥 Commits

Reviewing files that changed from the base of the PR and between 332711c and ad9d557.

📒 Files selected for processing (1)

• prmi/src/index/spectrum.rs

[nh13]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.