Skip to content

Default highlightable? to false for returnable: false fields#1131

Open
jwils wants to merge 1 commit intomainfrom
highlight-returnable-false-defaulting
Open

Default highlightable? to false for returnable: false fields#1131
jwils wants to merge 1 commit intomainfrom
highlight-returnable-false-defaulting

Conversation

@jwils
Copy link
Copy Markdown
Collaborator

@jwils jwils commented Apr 15, 2026

Summary

  • highlightable? now returns false for returnable: false fields when no explicit highlightable: is set, providing storage savings by default since non-returnable fields don't need to remain in _source for highlighting
  • source_excludes_paths now checks highlightable? when deciding what to exclude from _source, so explicitly highlightable: true fields stay available for search highlight snippets even when returnable: false
  • Adds highlightable: true to the test schema's internal_name and internal_details widget fields to demonstrate the explicit opt-in pattern
  • Adds acceptance test covering filtering, sorting, grouping, aggregating, and highlighting of hidden (returnable: false, highlightable: true) fields

Why only highlightable?

highlightable? is the only -able? predicate where returnable: false creates a meaningful conflict. The other predicates all operate on indexed data (inverted index / doc values), which is always present regardless of _source settings:

  • sortable? — Sorting uses doc values, not _source
  • filterable? — Filtering runs against the inverted index
  • groupable? — Grouping uses terms aggregations on indexed data
  • aggregatable? — Aggregations (min, max, avg, cardinality, etc.) use indexed data
  • sub_aggregatable? — Nested sub-aggregations also work on indexed data

Highlighting is unique because the datastore must read the original text from _source to produce contextual snippets (<em>matched text</em>). When returnable: false excludes a field from _source, highlighting becomes impossible — making it the only predicate where returnable: false should influence the default.

Test plan

  • All 1627 elasticgraph-schema_definition unit tests pass locally
  • All 1336 elasticgraph-graphql unit tests pass locally (1 pre-existing unrelated timezone failure)
  • Full CI build passes

🤖 Generated with Claude Code

When a field has `returnable: false` and no explicit `highlightable:`
setting, `highlightable?` now returns false. This provides storage
savings by default, since the primary reason for `returnable: false` is
to exclude field data from `_source`. Users can still opt in to
highlighting with `returnable: false, highlightable: true`.

Also updates `source_excludes_paths` to check `highlightable?` when
deciding what to exclude from `_source`, so that explicitly
highlightable fields remain available for search highlight snippets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jwils jwils force-pushed the highlight-returnable-false-defaulting branch from d874301 to f8854db Compare April 16, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant