Skip to content

docs: differentiator demo walkthrough + Act 2 robustness - #169

Merged
pdettori merged 4 commits into
rossoctl:mainfrom
pdettori:docs/differentiator-demo-walkthrough
Aug 26, 2026
Merged

docs: differentiator demo walkthrough + Act 2 robustness#169
pdettori merged 4 commits into
rossoctl:mainfrom
pdettori:docs/differentiator-demo-walkthrough

Conversation

@pdettori

@pdettori pdettori commented Aug 25, 2026

Copy link
Copy Markdown
Member

What

Adds serverless-harness-demo.md — a ~10-minute, differentiator-focused kind walkthrough with two acts — and wires it into the README so a new user can try the harness in minutes:

  • Act 1 — Durable resume across a true zero: cold-start a pod, scale to zero, resume the same session from a fresh pod that still remembers the policy (state lives in Redis, not the process).
  • Act 2 — A fleet from zero: fan five async leaves onto a Redis stream, watch KEDA materialize leaf-worker pods 0→N to drain them, collect verdicts, then collapse back to zero.

README Quick Start

Fleshes out the existing Quick Start with the three copy-paste bootstrap steps (clone --recurse-submodules, export credential, setup-kind.sh) matching the demo's Act 0, notes the default published-image pull (no local build) and default model, and points prominently at the two-act walkthrough as the guided tour.

Act 2 robustness

While running the walkthrough live, Act 2 produced no worker pods and 2e hung. Root cause: the leaf-worker ScaledJob wasn't applied (2a). The batch is still accepted onto the Redis stream, but with no ScaledJob, KEDA has nothing to scale — so no workers start in 2d and 2e polls forever. Everything else (KEDA, image, CRD, dispatch → lag=5 on the stream) was healthy. Applying the ScaledJob immediately spawned 5 workers, drained the queue, and returned the expected verdicts (auth/payments FLAGGED, rest CLEAR).

Two changes come out of that:

  1. Guard 2a — add kubectl get scaledjob leaf-worker right after the apply, plus a note that the check must pass before 2d and that a ScaledJob does not survive a kind delete/recreate. A skipped/failed 2a now fails loudly instead of silently no-op'ing.
  2. ttlSecondsAfterFinished: 60 on the leaf Job template — finished worker Jobs and their pods now GC themselves ~60s after completion: long enough to kubectl logs a worker for the 2e reveal, short enough that the fleet clears on its own after 2f instead of leaving Completed pods behind.

Verification

Ran the full walkthrough on a live kind-sh-knative cluster: Act 1 resume works across scale-to-zero; Act 2 spawns 5 leaf-workers, drains the stream (lag 5→0), returns correct verdicts, and collapses back to zero. ttlSecondsAfterFinished: 60 confirmed applied and reading back on the live ScaledJob.

🤖 Generated with Claude Code

Replace the toy secret-word walkthrough with a two-act demo that leads
with what a scale-to-zero agent does that an always-on one can't:

- Act 1 (durable resume): load a security policy via /turn, scale to a
  true zero, then resume the same session from a cold-started pod that
  still recalls the policy from Redis.
- Act 2 (fan-out from zero): dispatch five async leaves on /runs, watch
  KEDA materialize leaf-worker pods 0->N to drain the queue, collect the
  agent-driven FLAGGED/CLEAR verdicts, and watch the fleet collapse back
  to zero.

Everything runs on what setup-kind.sh installs; Act 2 adds one explicit
'kubectl apply -f leaf-scaledjob.yaml'. Install block matches the current
pull-by-default image behavior (rossoctl#154).

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
…d jobs

Act 2 silently no-ops if the leaf-worker ScaledJob isn't applied in 2a: the
batch is still accepted onto the Redis stream (lag grows), but KEDA has nothing
to scale, so no worker pods start in 2d and 2e polls the runs forever. Add a
`kubectl get scaledjob leaf-worker` check right after the apply so a
skipped/failed 2a fails loudly before 2d, and note that a ScaledJob does not
survive a `kind delete`/recreate.

Also set ttlSecondsAfterFinished: 60 on the leaf Job template so finished
worker Jobs and their pods garbage-collect ~60s after completion — long enough
to `kubectl logs` a worker for the 2e reveal, short enough that the fleet
clears on its own after 2f instead of leaving Completed pods behind.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
The Quick Start only linked to the walkthrough; a new user still had to open
the demo to find the bootstrap commands. Add the three copy-paste steps
(clone --recurse-submodules, export credential, setup-kind.sh) matching the
demo's Act 0, note the default published-image pull (no local build) and
model, then point prominently at serverless-harness-demo.md as the guided
two-act tour. Lets someone exploring the repo try the harness in minutes.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Paolo Dettori <dettori@us.ibm.com>

@pdettori pdettori left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs PR review — two non-blocking notes below; no must-fix issues.

Summary. Clean rewrite into a focused two-act walkthrough plus a runnable README Quick Start. The one functional change (ttlSecondsAfterFinished: 60) is correctly placed under jobTargetRef as a valid JobSpec field with a clear rationale comment. Nice bonus: the diff removes a leaked internal gateway URL (ete-litellm.bx.cloud9.ibm.com) and token placeholder in favor of generic ones. Verified against the repo: SH_IMAGE default, the claude-haiku-4-5 model, and the "KEDA installed but ScaledJob not" narrative in setup-kind.sh all check out.

Areas reviewed: Docs (README + demo), YAML (KEDA ScaledJob) · Agent/IDE config (.claude/.vscode): none · Commits: 3, all DCO-signed · CI: all green.

(Posted as a COMMENT review since GitHub doesn't allow self-approval — treat as an APPROVE-equivalent; both notes are optional.)

Comment thread serverless-harness-demo.md Outdated
Comment thread README.md Outdated
…l clone URL

Review notes on rossoctl#169:
- The intro table and Act 0 prose labeled the two acts A/B while every section
  header and cross-reference uses Act 1 / Act 2; relabel the table rows and the
  setup-kind.sh prose to 1/2 so readers no longer infer A→1, B→2.
- The clone URLs used the old kagenti org (resolves via GitHub's rename
  redirect) while the images are rossoctl; switch both the README Quick Start
  and demo Act 0 to the canonical rossoctl URL.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Paolo Dettori <dettori@us.ibm.com>
@pdettori
pdettori merged commit 186122b into rossoctl:main Aug 26, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant