Skip to content

fix(provider): retry or fall back after schema-invalid 2xx responses with no choices #5

Description

@Audacity88

Summary

A schema-invalid 2xx chat-completion response with no choices fails immediately as a non-retryable decode error. The configured retry count and fallback chain are not used, even though no output was emitted and the malformed response may be a transient proxy/model failure.

Reproduction

During a non-streaming free-model review run:

zoder exec --oneshot --approve none --no-stream \
  --max-tokens 4000 \
  --model nvidia/nemotron-3-ultra-550b-a55b:free \
  --require-free \
  < prompt.md

Zoder routed successfully through an OpenAI-compatible OpenRouter provider, attempted the real request, and returned:

zoder: error: malformed chat-completion response: no completion choices

The same route had completed successfully in an earlier run. No model output was emitted before the failure.

Current behavior

consume_full correctly rejects an empty choices array as ErrKind::Decode. However, ProviderError::retryable only treats timeout, rate-limit, server, and network errors as retryable. The one-shot dispatch loop therefore does not use its configured retries for this response shape.

This is correct fail-closed behavior for the individual response, but it turns a potentially transient empty proxy response into an immediate terminal transport failure.

Expected behavior

When a 2xx response has no meaningful completion choice and no output has been emitted, Zoder should make a bounded recovery attempt before failing the run. Possible behavior:

  1. Retry the same model/provider up to the configured retry count.
  2. If it remains schema-invalid, continue to the configured fallback chain when policy permits.
  3. Preserve the current terminal decode error when retries/fallbacks are exhausted.

The diagnostic should retain the selected provider/model and enough safe response metadata to distinguish a transient empty response from a permanently incompatible wire format without exposing credentials or arbitrary response content.

Environment

  • zoder 0.2.1
  • OpenAI-compatible OpenRouter provider
  • Model: nvidia/nemotron-3-ultra-550b-a55b:free
  • Non-streaming one-shot execution

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions