Skip to content

Fix skipped test: llm-coverage-evaluator "backup judge" (stale vs current DEFAULT_JUDGES) #48

Description

@nojibe

Problem

The "backup judge" test in src/cli/evaluators/__tests__/llm-coverage-evaluator.test.ts is broken and currently .skipped. Its expectations are stale relative to the current DEFAULT_JUDGES defined in src/cli/evaluators/llm-coverage-evaluator.ts, so it fails.

This test was failing on main before the Vitest migration too — it simply never ran in CI, so the breakage went unnoticed. During the Jest→Vitest migration (#45 / PR #46) it was marked .skip with an explanatory note rather than deleted, so no coverage decision is hidden. Now that unit suites run in CI (#33 / PR #47), this should be fixed and re-enabled.

Task

  • Review the current DEFAULT_JUDGES in src/cli/evaluators/llm-coverage-evaluator.ts and the backup-judge selection logic.
  • Update the test's expectations to match current behavior (or fix the source if the test is asserting the correct intended behavior and the source regressed — determine which).
  • Remove the .skip and confirm it passes under pnpm test:cli.

Acceptance criteria

  • The "backup judge" test is un-skipped and passing
  • Test reflects the actual intended backup-judge behavior (not just made to pass)
  • pnpm test:cli green

References

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions