Set libcudf kernel cache path - #398
Open
bdice wants to merge 1 commit into
Open
Conversation
simoneves
approved these changes
Aug 3, 2026
bdice
commented
Aug 3, 2026
|
|
||
| COPY velox-testing/presto/docker/launch_presto_servers.sh velox-testing/presto/docker/presto_profiling_wrapper.sh /opt | ||
|
|
||
| ENV LIBCUDF_KERNEL_CACHE_PATH=/var/lib/presto/data/libcudf-cache |
Contributor
Author
There was a problem hiding this comment.
This sets the environment variable for both GPU and CPU workers, because I didn't see an easy way to set it for only GPU workers. If you have alternative ideas, please feel free to push to this PR.
|
Note, @misiugodfrey is working on a fix by making a mountable write point, yet it has a different name than I expected. I was digging into this, and was going to provide a comment on PR #390: |
lamarrr
approved these changes
Aug 3, 2026
lamarrr
left a comment
lamarrr
left a comment
There was a problem hiding this comment.
Changes LGTM. Thanks for working on this!
6 tasks
misiugodfrey
added a commit
that referenced
this pull request
Aug 5, 2026
- Remove jq download block and PATH prefix from run_queries; replace remaining jq-based wait_for_workers_to_register with python3 - Add exchange.http-client.max-content-length=128MB to GPU coordinator to pair with exchange.max-response-size=64MB (avoids client rejecting responses it requested at SF30K+) - Remove exchange.max-response-size from GPU and CPU worker configs; native workers do not read this property (not in Configs.h) - Fix libcudf cache mount target: /var/lib/presto/data/libcudf-cache matches LIBCUDF_KERNEL_CACHE_PATH set by PR #398 in the native image - Port full per-iteration profiler structure (PROFILE_ITERATIONS env var, combined and per-iter modes) with CTAS integration; collect metrics per-iteration after each per-iter profiler stop - Add java/etc_coordinator/catalog/hive.properties override to preserve SOFT_AFFINITY for Java benchmarks after it was removed from the shared template - Record PRESTO_SESSION_PROPERTIES in benchmark_result.json context as session_properties; add field to BenchmarkMetadata; document format in _session_properties_from_env docstring; reject empty property names - Reject --iterations < 1 at session start - Set CPU worker exchange/sink buffers to 512MB (safe multi-worker default until generate_presto_config.sh reconciliation lands in #391); remove misleading comment promising generator behavior not in this PR - Remove CPU_* generator promise comment from cpu/etc_worker template
mattgara
approved these changes
Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Set
LIBCUDF_KERNEL_CACHE_PATH=/var/lib/presto/data/libcudf-cachein the native Presto image.Without an explicit path, cuDF uses the root user cache directory (
/root/.cache/libcudf). The Presto container filesystem is read-only, so that directory cannot be created. Older cuDF versions silently continued without a file cache. rapidsai/cudf#22975 makes failure to select or create a writable kernel-cache directory an error, which turns this into a startup failure.The Presto data directory is writable, so configure it as the kernel-cache location.