Harden evaluator evidence lifecycle and promotion workflow

[jon-devlapaz]

What changed

• make evaluator-owned skill instruction injection the explicit treatment intervention
• add a shared CodexRuntime lifecycle for target, rubric judge, pairwise judge, and calibration invocations
• isolate every invocation in a cleaned OS-temporary workspace outside retained output
• add promotion guardrails for independently controlled task paths, repeated trials, and calibration-bound rubric runs
• record activation, timing, token usage, infrastructure failures, and retained artifacts consistently
• harden task identifiers, payload hashing, runtime-home cleanup, and artifact links
• add a ten-case public React best-practices development benchmark
• update the evaluator contract, skill guidance, and Phase 2 plan

Why

The evaluator's scientific claim depends on holding the runner lifecycle constant while changing only the treatment. Target and judge paths had accumulated separate process and workspace behavior, leaving room for isolation drift and ancestor-project instruction leakage. Promotion intent was also not represented strongly enough in the executable contract.

This change concentrates the complete Codex invocation lifecycle behind one interface and makes the distinction between development evidence and promotion evidence explicit.

Impact

Operators get:

• deterministic treatment delivery without relying on model-side skill discovery
• identical isolation and cleanup policy across target and judging roles
• fail-fast infrastructure handling and visible invocation progress
• promotion-mode guardrails while preserving human ownership of holdout custody and transcript review
• a realistic public suite for development and regression work

The CLI and retained report workflow remain compatible. This does not claim that a visible development suite is an independent promotion holdout, and it does not treat a configured executable as an untrusted sandboxed process.

Validation

• python3 -m unittest discover -s tests -p 'test_*.py' — 47 tests passed
• ruff check skills/skill-eval-loop/scripts/skill_eval_loop.py tests/test_skill_eval_loop.py — passed
• python3 -m py_compile skills/skill-eval-loop/scripts/skill_eval_loop.py — passed
• skills/skill-eval-loop/scripts/healthcheck.sh — valid
• git diff --check — passed