fix: Fix retired claude-3-haiku-20240307 default causing sandbox run failures (#24)

[nojibe]

Summary

Fixes #24. Sandbox/playground runs that kept the default model list were failing generation with a 404 from Anthropic:

Anthropic API Error: 404 Not Found - {"type":"not_found_error","message":"model: claude-3-haiku-20240307"}

anthropic:claude-3-haiku-20240307 has been retired on the direct Anthropic API, and the client (src/lib/llm-clients/anthropic-client.ts) passes the model ID straight through to /v1/messages, so the call 404s. Because the failed generation produces no gradable output, the run's Macro Coverage Overview also showed 0%.

This model was hardcoded as a default in three places, so every run keeping the defaults hit it.

Changes

Replaced the retired anthropic:claude-3-haiku-20240307 with the current anthropic:claude-3-5-haiku-20241022 (already present in model-version-registry.ts) in the default model lists:

• src/app/api/sandbox/run/route.ts — DEFAULT_ADVANCED_MODELS
• src/app/api/v1/evaluations/run/route.ts — API default models
• src/app/sandbox/components/RunEvaluationModal.tsx — playground available + default model lists

Test plan

• Run a sandbox evaluation with the defaults and confirm the Haiku model returns a 200 and Macro Coverage populates (requires ANTHROPIC_API_KEY; not runnable in this environment).
• Verified by inspection: no remaining references to the retired model ID in the changed default lists; the replacement ID is a registered, current model.

Risks

• This is a targeted default swap; user-authored blueprints that explicitly pin the retired model would still fail. That broader, durable fix (proactive detection of retired models + graceful "deprecated" display instead of failing) is tracked in Research: proactively detect deprecated/retired models and surface results with a "deprecated" annotation instead of failing #32, which also captures a suggestion to route provider defaults through OpenRouter to decouple from any single provider's retirement schedule.

Related Issues

• Closes Weval.org Results Page Error #24
• Follow-up research: Research: proactively detect deprecated/retired models and surface results with a "deprecated" annotation instead of failing #32

🤖 Generated with Claude Code

https://claude.ai/code/session_01L1QemnhVU3wqvW8RPHcwx2

---

Generated by Claude Code