Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
995 changes: 68 additions & 927 deletions .github/workflows/pr.yaml

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions java/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,15 @@ per-entry artifact. The separate `java-gather` job downloads them (with
`merge-multiple: true`, so all subdirs land in a single parent dir), runs
Step 3, and uploads the combined `cudf_java_maven_repo` artifact.

### Step 4 - Smoke tests (optional, local)

After assembling a Maven-repo tree, run the packaging smoke suite under
[`java/smoke-tests/`](../smoke-tests/README.md):

```bash
./java/smoke-tests/bin/run.sh --maven-repo-dir /tmp/maven-repo
```

## Legacy: manual Dockerfile.rocky build (obsolete)

> The `java/ci/Dockerfile.rocky` + `java/ci/build-in-docker.sh` flow below is the
Expand Down
5 changes: 5 additions & 0 deletions java/smoke-tests/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES.
# SPDX-License-Identifier: Apache-2.0

.cache/
target/
81 changes: 81 additions & 0 deletions java/smoke-tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# cudf-java smoke tests

Smoke published `ai.rapids:cudf` classifier JARs under a Maven Central-style
classpath (cudf + slf4j only).

## Quick start

Exactly one source mode is required.

```bash
# Local gather tree (must contain ai/rapids/cudf/)
./java/smoke-tests/bin/run.sh --maven-repo-dir /tmp/cudf-java-build/maven-repo

# GitHub Actions gather artifact
./java/smoke-tests/bin/run.sh \
--artifact-url 'https://github.com/rapidsai/cudf/actions/runs/<run>/artifacts/<id>'

# Installed into ~/.m2
./java/smoke-tests/bin/run.sh --use-maven-home

# Maven Central release (version required)
./java/smoke-tests/bin/run.sh --use-maven-central --version 25.12.0
```

Narrow to one CUDA major:

```bash
./java/smoke-tests/bin/run.sh --maven-repo-dir /tmp/maven-repo --cuda-version 12
```

## Source modes

| Mode | Flag |
|---|---|
| GitHub Actions artifact | `--artifact-url URL` |
| Local Maven-repo tree | `--maven-repo-dir DIR` |
| `~/.m2/repository` | `--use-maven-home` |
| Maven Central | `--use-maven-central` (requires `--version`) |

Shared options:

| Option | Meaning |
|---|---|
| `--version VER` | Pin `ai.rapids:cudf` version. Required for Central. For other modes, if omitted, exactly one version directory must exist under `ai/rapids/cudf/`. |
| `--cuda-version 12\|13` | Narrow classifiers to this CUDA major |

## Classifier selection

Without `--cuda-version`, by host arch:

- `x86_64`: `unclassified`, `cuda12`, `cuda13`
- `aarch64`: `cuda12-arm64`, `cuda13-arm64`

With `--cuda-version N`:

- `x86_64` + 12: `unclassified`, `cuda12`
- `x86_64` + 13: `cuda13`
- `aarch64` + N: `cudaN-arm64`

Missing JARs are warned and skipped. The run fails if no classifier is actually smoke tested, or if any smoke-tested classifier fails.

After each classifier run, `run.sh` also requires the native `CUDF_LOG_WARN` from
`parquetChunkedLoggerSmoke` (`Chunked Parquet reader: a chunk_read_limit`) in the
captured smoke log under `.cache/logs/`.

## Artifact downloads

`--artifact-url` uses [`bin/fetch_bundle.sh`](bin/fetch_bundle.sh) (requires authenticated `gh`).
Downloads land at:

```text
java/smoke-tests/.cache/downloads/runs/<run_id>/artifacts/<artifact_id>/
```

(or under `$CUDF_JAVA_SMOKE_OUTPUT_DIR`). If that destination already exists and
contains `ai/rapids/cudf/`, the fetch warns and reuses it.

## Docker images

`docker/Dockerfile.cuda{12,13}` (Ubuntu 24.04 CUDA runtime + OpenJDK 17 + Maven).
Built on first use; rebuild with `docker build` / `docker rmi` if the Dockerfile changes.
111 changes: 111 additions & 0 deletions java/smoke-tests/bin/fetch_bundle.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
#!/usr/bin/env bash
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Download a java-gather GitHub Actions artifact into:
# <output-dir>/runs/<run_id>/artifacts/<artifact_id>/
# Default <output-dir>: java/smoke-tests/.cache/downloads

set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SMOKE_TESTS_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"

ARTIFACT_URL=""
OUTPUT_DIR="${CUDF_JAVA_SMOKE_OUTPUT_DIR:-${SMOKE_TESTS_ROOT}/.cache/downloads}"

info() { printf '==> %s\n' "$*" >&2; }
warn() { printf 'WARNING: %s\n' "$*" >&2; }
die() { printf 'ERROR: %s\n' "$*" >&2; exit 1; }

require_value() {
local flag="$1"
local value="${2:-}"
if [[ -z "${value}" ]]; then
die "${flag} requires a value"
fi
}

print_help() {
cat << EOF
Usage: $(basename "$0") --artifact-url <url> [--output-dir DIR]

Download the cudf_java_maven_repo GitHub Actions artifact into:
<output-dir>/runs/<run_id>/artifacts/<artifact_id>/

--output-dir DIR Default: ${SMOKE_TESTS_ROOT}/.cache/downloads
(or \$CUDF_JAVA_SMOKE_OUTPUT_DIR)

Requires gh. If the destination already exists and looks valid, warns and reuses it.
EOF
}

main() {
while [[ $# -gt 0 ]]; do
case "$1" in
-h|--help) print_help; exit 0 ;;
--artifact-url)
require_value "$1" "${2:-}"
ARTIFACT_URL="$2"
shift 2
;;
--output-dir)
require_value "$1" "${2:-}"
OUTPUT_DIR="$2"
shift 2
;;
*) die "Unknown argument: $1 (try --help)" ;;
esac
done

if [[ -z "${ARTIFACT_URL}" ]]; then
die "--artifact-url is required (try --help)"
fi
if ! command -v gh >/dev/null 2>&1; then
die "'gh' (GitHub CLI) is required"
fi
if ! command -v unzip >/dev/null 2>&1; then
die "'unzip' is required"
fi

local repo run_id artifact_id
if [[ "${ARTIFACT_URL}" =~ github\.com/([^/]+/[^/]+)/actions/runs/([A-Za-z0-9_]+)/artifacts/([A-Za-z0-9_]+)/?$ ]]; then
repo="${BASH_REMATCH[1]}"
run_id="${BASH_REMATCH[2]}"
artifact_id="${BASH_REMATCH[3]}"
else
die "Could not parse artifact URL: ${ARTIFACT_URL}"
fi

local dest="${OUTPUT_DIR}/runs/${run_id}/artifacts/${artifact_id}"
if [[ -d "${dest}/ai/rapids/cudf" ]]; then
warn "Destination already exists; reusing: ${dest}"
printf '%s\n' "${dest}"
return 0
fi
if [[ -e "${dest}" ]]; then
die "Destination exists but is incomplete (no ai/rapids/cudf/): ${dest}"
fi

mkdir -p "${dest}"
info "Downloading artifact ${artifact_id} from ${repo} run ${run_id}"
gh api \
-H "Accept: application/vnd.github+json" \
"/repos/${repo}/actions/artifacts/${artifact_id}/zip" \
> "${dest}/artifact.zip"
unzip -q -o "${dest}/artifact.zip" -d "${dest}"

if [[ -d "${dest}/cudf_java_maven_repo/ai" ]]; then
mv "${dest}/cudf_java_maven_repo/ai" "${dest}/"
rm -rf "${dest:?}/cudf_java_maven_repo"
fi
if [[ ! -d "${dest}/ai/rapids/cudf" ]]; then
die "No ai/rapids/cudf/ under ${dest} after download"
fi

printf 'artifact-url:%s\n' "${ARTIFACT_URL}" > "${dest}/.source"
info "Destination: ${dest}"
printf '%s\n' "${dest}"
}

main "$@"
Loading
Loading