Skip to content

fix(search): split FTS query terms on path separators#38

Merged
solomonneas merged 2 commits into
masterfrom
fix/fts-path-tokenization
Jul 17, 2026
Merged

fix(search): split FTS query terms on path separators#38
solomonneas merged 2 commits into
masterfrom
fix/fts-path-tokenization

Conversation

@solomonneas

Copy link
Copy Markdown
Contributor

What

fts_query quoted each whitespace-separated term whole, so a path-bearing query like change src/store/sync.rs became one FTS token that matched nothing. Terms now split on non-alphanumeric separators (keeping _ and $) into quoted tokens joined so any can match; tokens under 2 bytes drop; the existing LIKE fallback still catches degenerate queries.

Why

The co-change bench (receipts in benchmarks/context-cochange/) seeds personalized context packs with change <path> tasks. Path-bearing seeds matched zero symbols, so packs silently fell back to tiny unpersonalized output (646/230 chars). After the fix, recall at the same 4,000-char budget went 0.239 to 0.692 (Go corpus) and 0.200 to 0.504 (Rust corpus), both ahead of an editor-integrated repo-map baseline at matched budget (0.269 / 0.236). TypeScript remains near-zero for both surfaces (import-driven co-change, recorded as the next-experiment seed).

Verification

cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings, cargo test --all-features, cargo build --release (Brigade receipts). Two new tests fail against the old code. Post-fix bench rerun receipted in the results file.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • coderabbit-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: escoffier-labs/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 735eddb7-6d1f-44d0-8f3c-fde9cfe9e651

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/fts-path-tokenization

Comment @coderabbitai help to get the list of available commands.

solomonneas and others added 2 commits July 17, 2026 17:24
Path-style task queries were fusing file segments into one unusable
token. Split each whitespace term on non-alphanumeric boundaries,
drop sub-2-char noise tokens, and add regression tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@solomonneas
solomonneas force-pushed the fix/fts-path-tokenization branch from 2ed2d8c to 5f00da2 Compare July 17, 2026 21:24
@solomonneas
solomonneas merged commit 132c899 into master Jul 17, 2026
10 checks passed
@solomonneas solomonneas self-assigned this Jul 17, 2026
@solomonneas solomonneas mentioned this pull request Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant