Skip to content

Fix custom model objects with multiple keys being silently skipped#54

Merged
nojibe merged 1 commit into
mainfrom
claude/github-issue-22-qjdk1m
Jul 1, 2026
Merged

Fix custom model objects with multiple keys being silently skipped#54
nojibe merged 1 commit into
mainfrom
claude/github-issue-22-qjdk1m

Conversation

@nojibe

@nojibe nojibe commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #22. Custom model definitions using object syntax with multiple fields (e.g. { id, url, modelName }) were silently skipped by resolveModelCollections and logged a confusing warning.

Root cause

In src/cli/commands/run-config.ts, the object branch checked for a single-key provider object ({ "provider": "modelName" }) before the custom-model branch. Since a custom model has multiple keys, it failed the single-key check and was skipped with an Invalid object entry... Must have exactly one key warning. The else if (... && modelEntry.id) branch that was meant to handle custom models was unreachable — the preceding else if already matched every non-array object.

Changes

  • Restructured into a single object branch that checks modelEntry.id first (custom model definition → passed through directly), falling back to the provider-shorthand path. This eliminates the ordering dependency rather than just reordering two sibling branches, so the skip can't silently reappear.
  • Clarified the skip warning to mention the id option, so the failure is diagnosable instead of cryptic.
  • Added two regression tests: the exact { id, url, modelName } case from the issue, and a mixed case (custom model + string literal + collection placeholder).

Test plan

  • pnpm exec vitest run src/cli/commands/run-config.test.ts — 22 passed (including the 2 new tests, which return [] / drop the model under the old code).
  • tsc --noEmit clean for the touched files.

Related issues

Fixes #22

🤖 Generated with Claude Code


Generated by Claude Code

resolveModelCollections checked for a single-key provider object before
the custom-model (`id`) branch, so a custom model like
`{ id, url, modelName }` failed the single-key check and was skipped with
a confusing "Must have exactly one key" warning — the `.id` branch was
unreachable dead code.

Restructure to a single object branch that checks `id` first (custom
model definition, passed through) and falls back to provider shorthand.
Also clarify the skip warning to mention the `id` option so the failure
is diagnosable.

Fixes #22

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUWffHt992rYTqivjW2V8e
@railway-app railway-app Bot temporarily deployed to weval / app-pr-54 July 1, 2026 18:47 Destroyed
@railway-app

railway-app Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

🚅 Deployed to the app-pr-54 environment in weval

Service Status Web Updated (UTC)
weval-app ✅ Success (View Logs) Web Jul 1, 2026 at 6:54 pm

@nojibe nojibe merged commit 51dc4d3 into main Jul 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Custom model objects with multiple keys are silently skipped in resolveModelCollections

2 participants