feat(deploy): add OpenShift setup script (setup-ocp.sh)

[pdettori]

Closes #41.

Stacked on #42 (harness image fix). Review/merge #42 first; this PR's base will retarget to main once #42 lands. The diff shown here is the setup-script work only.

What

deploy/knative/setup-ocp.sh — the OpenShift-native sibling of setup-kind.sh, targeting OpenShift 4.20+. Base bring-up: OpenShift Serverless (Knative + Kourier), Redis, sandbox, leaf-work PVC, LLM secret, and the harness Knative Service, reachable over its auto-created Route.

Manifests stay shared with Kind via a small deploy/knative/overlays/ocp kustomize overlay (OCP tweaks are patches, not forked YAMLs).

How it differs from Kind (issue #41)

Concern	OpenShift approach
Knative	Red Hat OpenShift Serverless Operator (OLM Subscription) + KnativeServing CR (Kourier bundled)
Config	feature flags + autoscaler tuning in the CR spec (the operator reverts config-* ConfigMap patches)
Ingress	auto-created OpenShift Route (ksvc status.url); no Kourier port-forward
Harness UID	keeps runAsUser 65532; SA granted nonroot-v2 SCC so the explicit non-root UID is admitted (the GHCR image declares no USER) — issue #41 item #4b
Sandbox	pre-baked sandbox.Dockerfile (USER 65532), built in-cluster against the internal registry; also published to GHCR by build.yaml
Image	published GHCR image via --image (no dev.local/kind load)

Flags: --help, --dry-run, --image, --namespace, --sandbox-image, --skip-sandbox-build, --serverless-channel, --skip-keda. Idempotent (safe to re-run).

Scope

Base bring-up only. KEDA (async leaf, via the Custom Metrics Autoscaler Operator) and the optional Redis Enterprise Operator are deferred follow-ups — --skip-keda is the default. Redis stays the lightweight in-repo Deployment.

CI

build.yaml is extended (matrix) to build & push the sandbox image (ghcr.io/kagenti/serverless-harness-sandbox) the same way as the harness image.

Verification (live, OpenShift 4.20.8)

• ✅ OpenShift Serverless Operator install (idempotent) + KnativeServing Ready with the PVC/securityContext feature flags & autoscaler tuning in the CR
• ✅ nonroot-v2 SCC grant; harness pod runs non-root (was CreateContainerConfigError: runAsNonRoot before the fix)
• ✅ in-cluster sandbox build → internal registry; sandbox pod Running under restricted-v2
• ✅ leaf-work PVC binds (gp3-csi, RWO)
• ✅ ksvc/serverless-harness Ready; GET /health → HTTP 200 over the Route; POST /turn routes and creates a session
• ✅ idempotent re-run (operator install skipped)

A full /turn inference requires egress to the LLM gateway, which is environment-specific (this cluster couldn't reach the private gateway). bash -n + shellcheck clean; oc kustomize overlay renders.

Storage caveat

leaf-work is ReadWriteOnce. On block storage (AWS EBS gp3-csi) that binds to a single node — fine for the single harness consumer in base bring-up. Concurrent multi-node scale-out or co-mounting with the leaf-orchestrator would need RWX (documented in SMOKE.md).

Assisted-By: Claude Code

[pdettori]

Update: OpenShift smoke drivers + opt-in KEDA (commit 3ea4696)

Two capabilities added on top of the base bring-up:

Smoke on OpenShift. lib.sh/smoke.sh now support a Route mode — export KSVC_URL and the Kind smoke/experiment drivers run against the OCP Route (no Kourier port-forward, no Host header, -k for the router cert). Kind behavior is unchanged.

KSVC_URL=$(oc get ksvc serverless-harness -n default -o jsonpath='{.status.url}') ./deploy/knative/smoke.sh

Live result on OCP 4.20.8: 4/6 claims PASS — health, scale-to-zero, scale-up-from-zero, 404, and Redis session recall across a cold start (matching sessionId). The 2 failing claims assert on the LLM /turn response text, which needs cluster egress to the Anthropic gateway (environment-specific, not a harness defect).

KEDA opt-in. --with-keda (+ --keda-channel) installs the Red Hat Custom Metrics Autoscaler Operator (openshift-keda Subscription + KedaController CR) for the async-leaf ScaledJob path; default stays --skip-keda. Verified live: KedaController → Installation Succeeded, all openshift-keda deployments Available, scaledjobs.keda.sh CRD present. (Wiring/verifying the async-leaf ScaledJob itself on OCP is a further step.)

Assisted-By: Claude Code

[pdettori]

Update: OpenShift install guide (commit ce3a9a6)

Added deploy/knative/README-ocp.md — a full OpenShift install guide: prerequisites, setup-ocp.sh quick start + flag reference, what it installs, Route access, smoke-on-OCP, --with-keda, storage/SCC notes, image delivery (GHCR or in-cluster build), troubleshooting, and cleanup.

Also: slimmed the SMOKE.md OpenShift section to the smoke usage + a pointer, and linked the guide from README.md (Quick Start note, ToC, Documentation).

Assisted-By: Claude Code

[pdettori]

(Posted as COMMENT, not APPROVE — GitHub disallows approving your own PR. Findings are approve-level: no blocking issues.)

Adds setup-ocp.sh, the OpenShift-native sibling of setup-kind.sh (OCP 4.20+): OpenShift Serverless operator + KnativeServing CR, pre-baked sandbox image, nonroot-v2 SCC grant, shared manifests via an overlays/ocp kustomize overlay, plus Route-mode support in the smoke drivers (KSVC_URL) and docs. Clean shell (set -euo pipefail, dry-run, idempotent, secrets never echoed), correct set -u-safe empty-array curl expansion, and the OpenShift arbitrary-UID pattern (chgrp 0 / g=u / USER 65532) is right.

Verified the non-default --namespace path holds: every base YAML embeds namespace: default (redis, sandbox, PVC, service, SA, Role, RoleBinding subject) and redis://redis.default.svc:6379, so both render_overlay sed rewrites catch them all. #42 is merged, so this diff is clean against main.

Only two optional nits below — nothing blocking.

---

Author: pdettori (MEMBER — maintainer) - Areas: CI, Shell, Dockerfile, Kustomize, Docs - Commits: 3, all DCO-signed - CI: 9/9 passing

Assisted-By: Claude Code

[pdettori]

nit: the sandbox image builds with context: . (whole repo), but sandbox.Dockerfile has no COPY — a scoped context (e.g. deploy/knative) would ship a smaller build context. Purely cosmetic; the built image is identical either way.

[pdettori]

Done in 81bf425 — the sandbox build now uses context: deploy/knative (file stays deploy/knative/sandbox.Dockerfile, which build-push-action resolves from the workspace root). Verified locally that the image builds identically with the scoped context. Thanks!

[pdettori]

nit: FROM alpine:3.20 is tag-pinned, not digest-pinned. hadolint is green and this matches the repo's existing convention, so optional — a @sha256:… digest would harden reproducibility if you later want a stricter supply-chain policy.

[pdettori]

Leaving this as tag-pinned for now, for two reasons: (1) consistency — the harness Dockerfile pins node:22-alpine by tag too, and hadolint is green, so digest-pinning just the sandbox would be a lopsided/false-hardening signal; (2) a correct multi-arch pin needs the manifest-list (index) digest, and build.yaml runs on main-push only (not PRs), so I can't CI-verify a digest change on this PR before merge. Happy to do digest-pinning as a dedicated, repo-wide supply-chain PR (both images, index digests) if you want to adopt that policy — just say the word.