Fix parquet hybrid scan dictionary query bugs - #23611
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe change updates Parquet dictionary filtering to skip empty dictionaries safely and synchronize predicate initialization before decoding. New tests cover three-literal OR filters with absent and present dictionary values. ChangesDictionary filtering
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cpp/tests/io/experimental/hybrid_scan_filters_test.cpp`:
- Around line 1891-1939: Add a test case alongside the existing multi-literal
filter cases in filter_row_groups_with_dictionaries using a non-ASCII UTF-8
value stored in a dictionary and the same value as a cudf::string_scalar
literal. Build the filter expression and reader options consistently with the
surrounding tests, then assert the expected row groups to exercise hashing and
equality for UTF-8 dictionary strings.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: aa7d5704-45ba-432b-b40a-43f1146a765e
📒 Files selected for processing (2)
cpp/src/io/parquet/experimental/dictionary_page_filter.cucpp/tests/io/experimental/hybrid_scan_filters_test.cpp
For a predicate with more than
MAX_INLINE_LITERALS(2) literals,query_dictionariesdecided a column chunk had no dictionary page by testing its cuco set for zero slots. The problem is thatmake_valid_extentrounds every capacity up to at least one bucket, so that test never fired and the chunk was instead probed against a set that was never built — every literal came back absent and the row group was wrongly pruned, dropping rows. Detect the empty dictionary from the chunk's decoded value count instead, which is what the few-literals kernel already does.Also add a missing synchronization between initializing and further writing to
results, as threads initializing could stomp on threads filling it below.Checklist