SkillOpt

SkillOpt is an agent skill for training, evaluating, and improving other agent skills as reusable external capabilities.

It turns a SKILL.md or similar agent procedure file into an optimization target:

1. create train and validation task suites
2. run baseline rollouts with the target agent/model fixed
3. analyze success and failure traces
4. propose small, controlled skill edits
5. accept candidates only when validation improves
6. export a clean best_skill.md for deployment

The skill is inspired by Microsoft Research's SkillOpt framing: train the procedure, not the model weights.

Contents

• SKILL.md: portable agent instructions
• scripts/skillopt.py: local run directory, rollout, scoring, validation gate, report, and export helper
• references/evaluation.md: JSONL task and scorer reference
• agents/openai.yaml: UI metadata for compatible agents

Quick Start

python3 scripts/skillopt.py init --skill path/to/SKILL.md --out skillopt_runs/my-skill
python3 scripts/skillopt.py validate-tasks skillopt_runs/my-skill/tasks/val.jsonl
python3 scripts/skillopt.py run \
  --tasks skillopt_runs/my-skill/tasks/val.jsonl \
  --skill skillopt_runs/my-skill/source_skill.md \
  --out skillopt_runs/my-skill/rollouts/val_baseline \
  --agent-command "hermes -s {skill_path} -z {prompt}"

Use any agent command that can load a candidate skill and return task output. The validation gate compares rollout directories and accepts only measured improvements.