Skip to content

fix: extract bodiless trait methods and stop binary content reaching chunks - #4

Merged
stephane-segning merged 2 commits into
mainfrom
fix/extraction-gaps
Aug 7, 2026
Merged

fix: extract bodiless trait methods and stop binary content reaching chunks#4
stephane-segning merged 2 commits into
mainfrom
fix/extraction-gaps

Conversation

@stephane-segning

@stephane-segning stephane-segning commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

1. Summary

This PR changes:

It solves:


2. Intent

The intent of this PR is:

To close the two extraction gaps found when this crate was pulled out of its monorepo, without trading either of them for a quieter regression.

On #1, the obvious fix is wrong. Adding function_signature_item to the classifier makes a trait declaration a definition — correct — but it also makes it compete for its own name in the resolver. The resolver is precision-favouring: several same-named candidates with no disambiguating qualifier are dropped, not fanned out. So a trait with exactly one impl would go from one candidate to two the moment declarations were indexed, and every call to that method would stop resolving. The missing-symbol fix would have silently deleted calls edges — a strictly worse bug, because nothing would fail and no test asserted those edges.

The rule this PR adopts: a declaration is a definition (node, method edge, chunk) but not a call target. A call dispatches to an implementation, never to a declaration. Classifier::is_call_target is the seam, and a_call_to_a_single_impl_trait_method_still_resolves_to_the_impl is the test that proves the regression does not happen.

On #2, the guard existed and was correct — it was just at one of two call sites. chunk_file held it; the graph-enabled walk never calls chunk_file. Putting it where the chunk is produced makes the bypass structurally impossible rather than fixing this one instance of it. It also runs in the walk before extraction, so the graph stops ingesting binary content too — guarding only the chunker would have left tree-sitter parsing garbage into :Symbol nodes.


3. Scope

In Scope

  • function_signature_item classification, and the call-target seam that keeps it out of resolution
  • A shared is_binary guard applied at chunk-production and in the walk
  • Regression tests for both, including the end-to-end resolver proof
  • The one intended golden change

Out of Scope

  • Other languages' equivalent shapes. Java interface methods and TypeScript method_signature/abstract members go through the tags.scm path, not interesting_node, and are left exactly as they are. All 6 other golden files are unchanged by this PR, which is the evidence they were not touched. Whether a tags-driven language has the same declaration-vs-implementation ambiguity was left open here — Classifier::is_call_target leaves Tagged as-is rather than guessing per-grammar. Adversarial review has since answered it: Java does have the identical bug, today. tree-sitter-java's tags.scm captures a bodiless method_declaration exactly like a concrete one, so a single-impl interface call is dropped as ambiguous. Confirmed pre-existing (byte-identical against main), so it is not a regression from this PR — tracked as Java interface methods: single-impl calls are dropped as ambiguous (the tags-path twin of #1) #5.
  • TypeScript interface_declaration chunking, which interesting_node also omits. Noticed while here; genuinely a different bug, not filed as part of these two.
  • Widening the NUL sniff beyond its 512-byte prefix window. That bound is pre-existing and now has a test documenting it as a decision rather than an accident.

4. Verification

I verified this change by:

  • Running automated tests
  • Running manual tests
  • Checking logs
  • Checking metrics
  • Testing error cases
  • Testing permissions/security behavior
  • Testing rollback or failure behavior, if relevant

Commands run:

cargo test --no-fail-fast
cargo fmt --all -- --check
cargo clippy --all-targets --all-features --locked -- -D warnings
cargo llvm-cov --locked --all-targets --summary-only
cargo test --features container-tests --no-fail-fast

Results:

$ cargo test --no-fail-fast
    11 test binaries, all ok, 0 failed
    lib: 136 passed (was 130)

$ cargo fmt --all -- --check
    (clean)

$ cargo clippy --all-targets --all-features --locked -- -D warnings
    (no warnings)

$ cargo llvm-cov --locked --all-targets --summary-only
    TOTAL  95.40% lines   96.43% functions   96.00% regions
    (was 93.76% lines; CI floor is 85)

The golden diff, hand-verified. sample-repo is the only fixture containing a Rust trait, and it always contained trait Shape { fn describe(&self) -> f64; } at line 31 with an impl at line 35. The regenerated golden is additions only:

+ node  src/shapes.rs#31:describe   label "describe()"   line 31
+ edge  src/shapes.rs#30:Shape --method--> src/shapes.rs#31:describe

Nothing was removed. The pre-existing impl --method--> src/shapes.rs#35:describe is untouched, and no calls edge changed — which is exactly the property the design in §2 exists to protect. The other 6 golden files (java-repo, javascript-repo, polyglot-repo, python-repo, tsx-repo, typescript-repo) are byte-identical — they are asserted by the 13 test functions in tests/language_goldens.rs, which is a different number and was originally miscited here as “13 goldens”.

Tests added:

signature_only_trait_method_is_a_node_with_a_method_edge
signature_only_trait_method_is_not_a_call_target
a_bodied_method_next_to_its_declaration_is_still_the_only_call_target
a_call_to_a_single_impl_trait_method_still_resolves_to_the_impl
signature_only_trait_method_is_chunked_like_a_bodied_one
is_binary_detects_a_nul_only_within_the_sniff_window
build_graph_path_rejects_a_nul_laden_blob_exactly_as_chunk_file_does   (inverted from the bug-documenting test)
binary_blob_is_kept_out_of_the_graph_too_not_just_the_chunks

5. Screenshots / Evidence

Add evidence here:

Adversarial review outcome. An independent reviewer drove the public API against hand-built fixtures on this branch and on a clean clone of main, to separate "introduced here" from "pre-existing". It could not break the is_call_target design for Rust (generic bounds, cross-file calls, two unrelated traits with a colliding method name, default-body-plus-override all resolve identically pre/post), and independently re-derived the golden diff as complete and correct. It produced two acted-on findings:

  1. Java has the same ambiguity today — reproduced, confirmed pre-existing, filed as Java interface methods: single-impl calls are dropped as ambiguous (the tags-path twin of #1) #5.
  2. WalkStats had no counter for the new binary-skip path, so a rejected file vanished from the summary line entirely — a fair hit, since before this PR that file was counted into files_chunked (with contaminated content). Fixed in this PR: files_skipped_binary, logged and asserted.

It also caught a factual imprecision in this description ("13 language goldens" — 13 is the test-function count; there are 6 other golden files), corrected above.


6. Risk Assessment

Risk level:

  • Low
  • Medium
  • High

Potential risks:

  • More chunks than before. Every bodiless trait method now produces a chunk, so a trait-heavy repo gains chunks and therefore embedding cost at index time. This is the point of the fix, but it is a real cost change, not free.
  • More graph nodes than before, so graph_find_symbol will return declarations alongside implementations for the same name. Consumers that assumed one node per method name will now see two.
  • Binary files are now skipped entirely on the graph path. If any repo was relying on garbage chunks existing, they disappear. This is intended.
  • The declaration-is-not-a-call-target rule is a judgement call. It is the right default, but it means a call to a trait method whose impl lives outside the indexed repo now resolves to nothing rather than to the declaration.

Mitigation:

  • The golden proves the change is additive on a real fixture, and the 6 other golden files prove no other language moved.
  • The resolver behaviour is pinned by an explicit end-to-end test, so a future change that makes declarations call targets will fail loudly instead of quietly dropping edges.
  • Both previously-buggy behaviours were covered by tests asserting the bug; those tests are inverted, so a regression in either direction is caught.
  • The last risk is stated rather than mitigated — it is a deliberate trade, and the reviewer question in §8 asks about it directly.

7. AI Usage Declaration

AI was used for:

  • Understanding existing code
  • Generating code
  • Refactoring
  • Generating tests
  • Drafting documentation
  • Reviewing the diff
  • Not used

Human verification:

  • I understand every meaningful change in this PR
  • I checked generated code manually
  • I checked generated tests manually
  • I removed unsupported AI assumptions
  • I accept responsibility for this PR

Both bugs were independently reproduced with a minimal driver before any code was changed, and the golden diff was read line by line against the fixture source rather than regenerated and trusted.


8. Reviewer Focus

Please focus your review on:

  • Correctness
  • Architecture
  • Security
  • Performance
  • Tests
  • Maintainability
  • Product intent
  • Edge cases

Specifically:

  • Is "a declaration is a definition but not a call target" the right rule? The alternative — declarations as call targets — is more intuitive and strictly worse in practice for single-impl traits. A third option would be to prefer bodied definitions inside pick(), which is more precise but pushes language-specific knowledge into the language-agnostic resolver.
  • Should a call to a trait method with no in-repo impl fall back to the declaration? Today it resolves to nothing. That is a real loss for repos that implement external traits, and it would be a small change to pick().
  • The is_binary sniff is a 512-byte prefix. A file with clean text followed by a NUL at byte 600 still gets chunked. Pre-existing, now tested and documented — but worth deciding whether the bound is right rather than inherited.

…chunks

Closes #1 and #2 — two gaps in the extraction path, both pre-existing.

#1 Trait methods declared without a default body parse as
   `function_signature_item`, not `function_item`, and were classified as
   nothing at all — so a Rust trait interface contributed zero callable
   symbols to the graph AND no chunk to semantic search.

   They are now definitions: a node, a `method` edge from the trait, and a
   chunk, with the same `chunk_type` as a bodied function (a new value
   would leak into every consumer's stored data for no reader benefit).

   They are deliberately NOT call targets. The resolver is
   precision-favouring: several same-named candidates with no qualifier are
   dropped, not fanned out. Had declarations joined the candidate set, a
   trait with exactly one impl would have gone from one candidate to two
   and every call to that method would have stopped resolving — fixing a
   missing-symbol bug by silently deleting `calls` edges. A call dispatches
   to an implementation, never to a declaration.

#2 `chunk_file` rejected binary content via a NUL sniff, but the
   graph-enabled walk never calls it — it parses the tree itself and falls
   back straight to `chunk_text`/`window_chunks`, which had no guard. A
   binary blob that happens to be valid UTF-8 (NUL is a legal codepoint)
   was windowed with raw NUL bytes in `Chunk::content`, on the one path
   production runs. PostgreSQL's `text` rejects NUL outright, so this
   failed at persist time far from its cause.

   The guard is now a shared `is_binary` applied where a chunk is produced
   (`chunk_text`) and in the walk before either consumer — so the graph
   never ingests binary either, not just the chunker.

The sample-repo golden gains exactly one node and one `method` edge (the
`Shape::describe` declaration it always contained). Additions only: no edge
removed, no `calls` edge lost. Coverage 93.8% -> 95.4%.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: daeece8

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Raised by adversarial review of this PR: a file rejected by the new
is_binary guard incremented no counter at all, so it vanished from the
walk summary. Before this PR the same file WAS counted into files_chunked
(with NUL-contaminated content), so the fix traded a data bug for an
observability gap.

Without the counter, 'this repo has fewer indexable files than expected'
and 'a binary asset is misnamed with a source extension' look identical
from the summary line, and only the second is actionable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@stephane-segning
stephane-segning merged commit ddf3769 into main Aug 7, 2026
8 checks passed
@stephane-segning
stephane-segning deleted the fix/extraction-gaps branch August 7, 2026 02:10
stephane-segning added a commit that referenced this pull request Aug 8, 2026
Closes #5 — the tags-path twin of the Rust fix in #4. A Java interface method
(or abstract method) and a TypeScript interface/abstract-class member both
parse as an ordinary tagged definition, so they registered as call targets
under the same bare name as their implementation. A single-implementation
interface therefore had TWO same-named candidates and no qualifier to
disambiguate them, so the precision-favouring resolver dropped every call to
it as ambiguous.

`Classifier::is_call_target`'s `Tagged` arm now excludes a definition when
BOTH (a) its node kind is one of the specific shapes verified (via each
grammar's `node-types.json`) to be legitimately bodiless — Java
`method_declaration`, TypeScript `method_signature` /
`abstract_method_signature` / `function_signature` — AND (b) it has no
`body` child. The check is scoped to that node-kind allowlist rather than
applied to every `Tagged` definition, because several JS `tags.scm` patterns
anchor `@definition.function` on a wrapper node with no `body` field concept
of its own (`variable_declarator` for `const f = () => {}`,
`assignment_expression`, `pair`) even though the function value they wrap
always has a body — treating those as bodiless would have wrongly dropped
every const-arrow-function call target (caught by the `javascript-repo`
golden gaining a spurious drop before the list was narrowed).

Python is unaffected: `function_definition.body` is a required field even
for a `pass`/`...`-bodied abstract method, so there is no bodiless shape to
detect there.

Two new fixtures (`java-interface-repo`, `typescript-interface-repo`) prove
the fix end-to-end: a single-implementation interface method now resolves,
both via a qualified call (matching this suite's existing `Widget.build()`
house style) and a bare call. All six pre-existing per-language goldens are
byte-identical to before this change — confirmed via `git status` after
`UPDATE_GOLDEN=1`, not just re-running the comparison — so this is additive
only, no regression to already-resolving calls.

FINDING, not fixed here (out of scope per the resolver-ambiguity-policy
exclusion): the issue's own literal reproduction — calling through an
interface-typed variable, `g.greet()` — still does not resolve after this
fix. `resolve::pick`'s single-candidate branch rejects a candidate whose
`scope` doesn't textually equal the call's qualifier, and the tags path sets
that qualifier to the raw receiver identifier (`g`, the parameter name) —
there is no type inference, so a receiver variable can never textually match
the type that defines the method it calls. Documented with a dedicated test
(`java_call_through_an_interface_typed_variable_needs_a_qualifier_match`)
rather than silently worked around.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant