Skip to content

fix(ai): Mistral choices guard, Codex non-retryable errors, faux cache-write token count - #1647

Closed
yuzhiyang1 wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
yuzhiyang1:fix/ai-provider-stream-bugs
Closed

fix(ai): Mistral choices guard, Codex non-retryable errors, faux cache-write token count#1647
yuzhiyang1 wants to merge 1 commit into
PrimeIntellect-ai:mainfrom
yuzhiyang1:fix/ai-provider-stream-bugs

Conversation

@yuzhiyang1

@yuzhiyang1 yuzhiyang1 commented Aug 22, 2026

Copy link
Copy Markdown

Fixes three provider bugs found by code review.

Mistral: crash on choices-less chunks

packages/ai/src/providers/mistral.ts accessed chunk.choices[0] without a guard. Endpoints that emit usage-only chunks (choices missing, empty, or undefined — common when stream_options.include_usage is on) threw a TypeError and aborted the whole stream. Guard with Array.isArray(chunk.choices), matching the existing pattern in openai-completions.ts.

Codex: non-retryable HTTP errors were retried

In packages/ai/src/providers/openai-codex-responses.ts, when isRetryableError() classified an HTTP status (e.g. 400/401/403) as non-retryable, the code threw — but the throw sat inside the same try as the fetch, so it was caught by the loop's own catch, which retried unless the message happened to contain "usage limit". Non-retryable errors now set lastError and break, and are thrown once after the loop.

faux: cache-write tokens double-counted

In packages/ai/src/providers/faux.ts, input subtracted only cacheRead, so cached-write tokens stayed in input while also being counted in cacheWrite, inflating totalTokens. Subtract both, matching openai-completions.ts usage accounting. First-turn behavior is now input: 0, cacheWrite: promptTokens instead of counting the prompt twice.

Note

Fix Mistral choices guard, Codex non-retryable errors, and faux cache-write token count

  • In consumeChatStream in mistral.ts, guard chunk.choices[0] access so usage-only chunks that lack a choices array are skipped instead of throwing.
  • In streamOpenAICodexResponses in openai-codex-responses.ts, non-OK HTTP responses now set lastError and break the retry loop, so errors like 401/403 fail fast instead of being retried.
  • In withUsageEstimate in faux.ts, input tokens are now max(0, promptTokens - cacheRead - cacheWrite) in all cases, preventing cache-write tokens from being double-counted as input.
  • Risk: faux provider usage.input values change for sessions with prompt caching enabled; consumers relying on the old (inflated) input count will see lower numbers.

Macroscope summarized c240b2b.

…etryable errors, fix faux cache-write token double count
@github-actions

Copy link
Copy Markdown

Hi @yuzhiyang1, thanks for your interest in contributing!

This project requires that pull request authors are vouched, and you are not in the list of vouched users.

This PR will be closed automatically. See https://github.com/PrimeIntellect-ai/prime-agent/blob/main/CONTRIBUTING.md for more details.

@github-actions github-actions Bot closed this Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant