Skip to content

feat: add train-model and demonstrate-value skills, exponential backoff in babysit-training#6

Merged
lawwu merged 1 commit into
mainfrom
feat/issues-2-3-5-train-model-demonstrate-value-backoff
Mar 15, 2026
Merged

feat: add train-model and demonstrate-value skills, exponential backoff in babysit-training#6
lawwu merged 1 commit into
mainfrom
feat/issues-2-3-5-train-model-demonstrate-value-backoff

Conversation

@lawwu

@lawwu lawwu commented Mar 14, 2026

Copy link
Copy Markdown
Owner

Closes #2, #3, #5

Summary

  • Add exponential back off to babysit-training #2 — Exponential backoff in babysit-training: adds --max-interval and --no-backoff flags; interval grows 1.5× per uneventful poll (capped at --max-interval, default 300s) and resets on anomaly, >5% loss delta, new epoch, or Vertex AI state change; progress output now shows next poll in Xs
  • Add an explicit training skill and also add it to orchestrate_e2e and benchmark_e2e #3train-model skill: new action skill that parses and launches a training command, applies HP overrides, delegates monitoring to babysit-training, enforces early stopping, tracks checkpoints, and invokes check-failed-run on failure; orchestrate-e2e gate 6 now calls train-model instead of running training inline
  • add skill /demonstrate-value #5demonstrate-value skill: new post-eval skill that reads check-eval.json, explain-model.json, and build-baseline.json, translates metrics into business-language impact estimates, and renders a stakeholder-ready HTML presentation via showboat; added as an optional step in orchestrate-e2e after gate 8

Files changed

  • skills/babysit-training/SKILL.md — backoff logic, new flags, updated example
  • skills/train-model/SKILL.md — new skill
  • skills/demonstrate-value/SKILL.md — new skill
  • skills/orchestrate-e2e/SKILL.md — gate 6 → train-model; optional demonstrate-value step
  • skills/orchestrate-e2e/references/lifecycle-gates.md — gate 6 evidence updated
  • skills/orchestrate-e2e/references/artifact-contract.md — gate numbering fixed; train-model.json and demonstrate-value artifacts added
  • skills/benchmark-e2e/references/skill-matrix.md + modes.md — stage 6 updated to train-model
  • references/schemas.md — train-model and demonstrate-value schemas; orchestrate-e2e gate_name enum updated
  • references/vocabulary.md — Terminal State enum added
  • README.md — train-model and demonstrate-value rows added (14 skills total)
  • plugin.json — bumped to 0.2.0

Test plan

  • claude --plugin-dir ./plugins/agentic-ml — verify train-model and demonstrate-value appear in skill list
  • Invoke /train-model with a toy training command and confirm it delegates to babysit-training
  • Invoke /babysit-training with --interval 5 --max-interval 30 on a long-running process and confirm interval grows and resets on anomaly
  • Invoke /demonstrate-value with existing check-eval.json and confirm HTML is generated

…ff in babysit-training

Closes #2, #3, #5

- babysit-training: add exponential backoff (1.5x per uneventful poll,
  capped at --max-interval, reset on anomaly/loss spike/state change);
  new --max-interval and --no-backoff flags; show next-poll interval in
  progress output
- train-model: new skill wrapping babysit-training for monitoring and
  check-failed-run for failure recovery; supports HP overrides, early
  stopping, and checkpoint management
- demonstrate-value: new skill generating a stakeholder-ready HTML
  presentation via showboat from eval/explainability/baseline artifacts
- orchestrate-e2e: gate 6 delegates to train-model; optional
  demonstrate-value step added after gate 8
- schemas.md/vocabulary.md: add train-model and demonstrate-value
  schemas, Terminal State enum
- lifecycle-gates, artifact-contract, skill-matrix, modes: updated
  references for train-model at stage 6
- plugin.json: bump to 0.2.0
@lawwu
lawwu merged commit 51fc655 into main Mar 15, 2026
1 check passed
@lawwu
lawwu deleted the feat/issues-2-3-5-train-model-demonstrate-value-backoff branch March 15, 2026 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add exponential back off to babysit-training

1 participant