Record what a run was, not just what it printed

[Rebreda]

The gap

listenr eval has nine flags and not one writes to disk. Every WER figure and keyword table is stdout only. listenr finetune keeps metrics via the HF trainer (trainer_state.json, tensorboard) but records nothing about its own invocation, so an adapter directory cannot tell you what produced it.

This bit for real: a three-way keyword comparison (base 22% / base+prompt 62% / fine-tuned 13% recall) briefly existed only as terminal scrollback after an ad-hoc capture attempt died with PermissionError on a read-only container mount. And #26 made splits comparable; comparability is worthless if the comparison is never recorded.

What this adds

listenr eval --output results.json — everything printed, plus what the printout compresses away:

{
  "listenr_version": "0.1.1.dev5+...",
  "created_utc": "2026-08-25T23:45:15+00:00",
  "mode": "split",
  "model": "/data/merged",
  "base_model": "openai/whisper-tiny",
  "dataset": "/data/dataset/hf_dataset",
  "split": "test",
  "n_requested": 8,
  "n_evaluated": 8,
  "wer_normalization": "whisper_english",
  "wer_pct": {"fine_tuned": 13.86, "base": 11.98},
  "keyword_recall": {"fine_tuned": {"technology": {"hits": 3, "expected": 3}}},
  "clips": [
    {"uuid": "...", "audio_path": "...", "duration_s": 18.26,
     "reference": "...", "hypotheses": {"fine_tuned": "...", "base": "..."},
     "keywords": {"fine_tuned": {"technology": true}}}
  ]
}

The per-clip array is the point: aggregates say whether something moved, clip rows say why, and they allow re-scoring under a different normalization without re-running inference. Works in split mode and --audio single mode. Keyword tallies become named fields (hits/expected) — a results file should not require knowing that index 0 means hits.

listenr finetune always writes run.json beside the adapter — resolved args, base model, architecture, dataset path and split sizes, trainable-param count, accelerator line. Written with status: "started" before the first step so a crashed run still leaves evidence, rewritten "completed" at the end. With trainer_state.json it answers both questions an adapter dir raises: what produced this, and how did it go.

Design notes

• New finetune/report.py is pure (no torch), like the other tested helpers; 32 new tests, full suite 381 green.
• Writes are atomic (tmp+replace, same as the manifest writer): a crash mid-write cannot destroy a previous good record.
• Internal label merged maps to fine_tuned in the file — keys in a results file outlive the code's terminology.
• WER rounded to two decimals; the third is false precision at these sample sizes.
• Docs: new "Keeping the results" section in finetune-amd.md, including the tensorboard logs the trainer already writes by default.

Verified on hardware

Live eval on the gfx1151 ROCm container (--n 8 --compare-base --keyword technology --keyword phone --output /data/out/results.json) wrote the report to a mounted volume, including the exact scenario that motivated this.

🤖 Generated with Claude Code