Skip to content

Fix parquet hybrid scan dictionary query bugs - #23611

Open
pmattione-nvidia wants to merge 3 commits into
NVIDIA:mainfrom
pmattione-nvidia:fix_pq_hybrid_dict_sync
Open

Fix parquet hybrid scan dictionary query bugs#23611
pmattione-nvidia wants to merge 3 commits into
NVIDIA:mainfrom
pmattione-nvidia:fix_pq_hybrid_dict_sync

Conversation

@pmattione-nvidia

@pmattione-nvidia pmattione-nvidia commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

For a predicate with more than MAX_INLINE_LITERALS (2) literals, query_dictionaries decided a column chunk had no dictionary page by testing its cuco set for zero slots. The problem is that make_valid_extent rounds every capacity up to at least one bucket, so that test never fired and the chunk was instead probed against a set that was never built — every literal came back absent and the row group was wrongly pruned, dropping rows. Detect the empty dictionary from the chunk's decoded value count instead, which is what the few-literals kernel already does.

Also add a missing synchronization between initializing and further writing to results, as threads initializing could stomp on threads filling it below.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@pmattione-nvidia pmattione-nvidia self-assigned this Aug 10, 2026
@pmattione-nvidia pmattione-nvidia added bug Something isn't working improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 10, 2026
@copy-pr-bot

copy-pr-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the libcudf Affects libcudf (C++/CUDA) code. label Aug 10, 2026
@pmattione-nvidia
pmattione-nvidia marked this pull request as ready for review August 10, 2026 20:29
@pmattione-nvidia
pmattione-nvidia requested a review from a team as a code owner August 10, 2026 20:29
@pmattione-nvidia pmattione-nvidia removed the bug Something isn't working label Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved dictionary-based filtering for empty dictionaries.
    • Ensured reliable evaluation of fixed-width literal filters during parallel processing.
    • Corrected filtering behavior for OR expressions with multiple literals, including row groups with and without dictionary pages.
  • Tests

    • Added coverage for retaining or pruning row groups based on dictionary contents.

Walkthrough

The change updates Parquet dictionary filtering to skip empty dictionaries safely and synchronize predicate initialization before decoding. New tests cover three-literal OR filters with absent and present dictionary values.

Changes

Dictionary filtering

Layer / File(s) Summary
Dictionary query and decoding updates
cpp/src/io/parquet/experimental/dictionary_page_filter.cu
query_dictionaries computes value counts before hash-set probing and skips empty dictionaries. evaluate_few_fixed_width_literals adds synchronization before dictionary decoding.
Multi-literal OR coverage
cpp/tests/io/experimental/hybrid_scan_filters_test.cpp
Tests cover three-literal OR filters where queried values are absent from or partially present in a row-group dictionary.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested labels: bug, cuIO

Suggested reviewers: vuule, bdice, mhaseeb123

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: fixing Parquet hybrid scan dictionary query bugs.
Description check ✅ Passed The description directly explains the dictionary detection bug, synchronization fix, regression tests, and affected predicates.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/tests/io/experimental/hybrid_scan_filters_test.cpp`:
- Around line 1891-1939: Add a test case alongside the existing multi-literal
filter cases in filter_row_groups_with_dictionaries using a non-ASCII UTF-8
value stored in a dictionary and the same value as a cudf::string_scalar
literal. Build the filter expression and reader options consistently with the
surrounding tests, then assert the expected row groups to exercise hashing and
equality for UTF-8 dictionary strings.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: aa7d5704-45ba-432b-b40a-43f1146a765e

📥 Commits

Reviewing files that changed from the base of the PR and between 463fba2 and a5133a2.

📒 Files selected for processing (2)
  • cpp/src/io/parquet/experimental/dictionary_page_filter.cu
  • cpp/tests/io/experimental/hybrid_scan_filters_test.cpp

Comment thread cpp/tests/io/experimental/hybrid_scan_filters_test.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant