fix: reset stale location counts in merge mode - #1
Merged
ajroetker merged 2 commits intoJul 24, 2026
Merged
Conversation
…rge mode A hasLocs=false posting left numLocValues/numFieldIDs holding the previous posting's counts. mergeTermFreqNormLocs infers hasLocs from NumLocationValues()>0, so the merge reader consumed location values that belonged to later postings until the chunk underflowed with 'read location values: EOF'. Any postings list where a term appears both in a term-vectored field and a plain field (numeric tokens, ids) hits this; the resulting merge failure is deterministic and retries forever. Query path is unaffected (it gates on hasLocs directly). Verified against four production segments from three environments: all fail merge before this change, all merge cleanly after; zapx test suite green (the three pre-existing arm64/NEON StreamVByte failures are unrelated and tracked separately).
Builds a segment where one term produces a postings list interleaving hasLocs=true and hasLocs=false postings, with another hasLocs=true posting after a false one, then merges it. Before the stale count reset this failed deterministically with 'read location values: EOF'; also verifies per-posting freq and location counts in the merged output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root cause
nextAtOrAfter only assigned the location counters for postings that had locations. A location-less posting therefore inherited the previous posting counters. mergeTermFreqNormLocs infers location presence from those counters and consumed values belonging to later postings until the chunk failed with read location values: EOF.
The failure is deterministic when the same term appears in both a term-vectored field and a plain field. Segment bytes are valid; this is a merge-reader state leak.
Validation
Related incident: antflydb/antfly#381