Skip to content

test(ai): select the copilot codex model dynamically in the tool-call-id handoff tests - #1356

Open
code-yeongyu wants to merge 1 commit into
mainfrom
test/dynamic-copilot-codex-model
Open

test(ai): select the copilot codex model dynamically in the tool-call-id handoff tests#1356
code-yeongyu wants to merge 1 commit into
mainfrom
test/dynamic-copilot-codex-model

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Why

Release-time typecheck keeps breaking on a retired github-copilot ModelId pin. publish-npm.yml -> scripts/release.mjs regenerates the model catalog live from models.dev, then runs tsc. A test that passes a literal id into getModel's ModelId parameter fails the moment upstream drops that id, even when the live test is skipIf-gated.

This class of failure hit senpi on 2026-08-28 (fireworks-models.test.ts:45) and again today: release run 33842146239 failed because gpt-5.2-codex left the live github-copilot catalog. #1353 unblocked 2026.9.4-2 by swapping the pin to gpt-5.3-codex. That is still a literal pin, so the next retirement repeats the outage.

What changed

  • packages/ai/test/tool-call-id-normalization.test.ts: the two getModel("github-copilot", ...) pins (and the openrouter openai/gpt-5.2-codex pins) now select a family match from getModels(...). No literal string flows into getModel's ModelId parameter.
  • If the catalog has no match, the helper throws github-copilot catalog has no codex-family model; update the test (and a sibling message for openrouter) instead of silently skipping.
  • Token-gated live handoff coverage is unchanged. A small non-gated unit test asserts the helper returns a /codex/ id on the committed catalog and throws that explicit message for an empty stub catalog.

Verification

  • bunx tsc --noEmit exit 0 on the committed catalog
  • npm --prefix packages/ai run generate-models (live) then bunx tsc --noEmit exit 0; regen discarded
  • bunx vitest run packages/ai/test/tool-call-id-normalization.test.ts green (2 passed, 4 skipIf'd without tokens)
  • bunx biome check --error-on-warnings packages/ai/test exit 0
  • node scripts/check-pr-changelog.mjs --base origin/main exit 0 (test-only; no runtime source)

Summary by cubic

Fixes release-time typecheck failures by selecting the Copilot Codex model dynamically in the tool-call-id handoff tests. The tests previously used hardcoded model IDs that would break when models.dev retired them; now they pick a codex-family model from the live catalog and throw a clear error if none exists.

Written for commit a0a43c5. Summary will update on new commits.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant