Problem
The "backup judge" test in src/cli/evaluators/__tests__/llm-coverage-evaluator.test.ts is broken and currently .skipped. Its expectations are stale relative to the current DEFAULT_JUDGES defined in src/cli/evaluators/llm-coverage-evaluator.ts, so it fails.
This test was failing on main before the Vitest migration too — it simply never ran in CI, so the breakage went unnoticed. During the Jest→Vitest migration (#45 / PR #46) it was marked .skip with an explanatory note rather than deleted, so no coverage decision is hidden. Now that unit suites run in CI (#33 / PR #47), this should be fixed and re-enabled.
Task
- Review the current
DEFAULT_JUDGES in src/cli/evaluators/llm-coverage-evaluator.ts and the backup-judge selection logic.
- Update the test's expectations to match current behavior (or fix the source if the test is asserting the correct intended behavior and the source regressed — determine which).
- Remove the
.skip and confirm it passes under pnpm test:cli.
Acceptance criteria
References
Problem
The "backup judge" test in
src/cli/evaluators/__tests__/llm-coverage-evaluator.test.tsis broken and currently.skipped. Its expectations are stale relative to the currentDEFAULT_JUDGESdefined insrc/cli/evaluators/llm-coverage-evaluator.ts, so it fails.This test was failing on
mainbefore the Vitest migration too — it simply never ran in CI, so the breakage went unnoticed. During the Jest→Vitest migration (#45 / PR #46) it was marked.skipwith an explanatory note rather than deleted, so no coverage decision is hidden. Now that unit suites run in CI (#33 / PR #47), this should be fixed and re-enabled.Task
DEFAULT_JUDGESinsrc/cli/evaluators/llm-coverage-evaluator.tsand the backup-judge selection logic..skipand confirm it passes underpnpm test:cli.Acceptance criteria
pnpm test:cligreenReferences
src/cli/evaluators/__tests__/llm-coverage-evaluator.test.tssrc/cli/evaluators/llm-coverage-evaluator.ts(DEFAULT_JUDGES)