Separate experiments from eval definitions - #1500

Merged

christso merged 18 commits into

mainfrom

experiments-separation

Jun 24, 2026

Commits on Jun 23, 2026

docs: plan experiment separation
christso
committed
17e1e22
feat: add experiment config foundation
christso
committed
ae79ac5
docs: reconcile repeat-run plan to experiment-level placement
christso
committed
4a7856a
feat: add native experiment execution support
christso
committed
bbdcead
fix: skip empty eval suites after experiment filtering
christso
committed
8ac7d34
feat: add experiment repeat schema and docs
christso
committed
362aa33
chore: format experiment repeat test
christso
committed
b242160
chore: format experiment schema
christso
committed
544981e
fix: preserve repeat trial artifact metadata
christso
committed
2c2505d
docs: require live experiment dogfood
christso
committed
a90002d

Commits on Jun 24, 2026

feat: write repeat attempts as run folders
christso
committed
7b245a2
feat: align repeat run artifacts with vercel layout
christso
committed
319652f
feat: flatten repeat case summary timing
christso
committed
cb5bff0
feat: remove execution metrics from grading artifacts
christso
committed
7910e08
feat: use answer artifact for run outputs
christso
committed
3726b76
feat: align repeat artifacts with task prompts
christso
committed
e5607f8
test: update rerun fixtures for answer artifacts
christso
committed
407e409
feat: embed repeat run metrics in result artifacts
christso
committed
159c0b6