Skip to content

Retry ModelBehaviorError so a truncated stream no longer kills the run#76

Open
inf-quantavius wants to merge 1 commit into
context-labs:mainfrom
inf-quantavius:retry-root-model-behavior-error
Open

Retry ModelBehaviorError so a truncated stream no longer kills the run#76
inf-quantavius wants to merge 1 commit into
context-labs:mainfrom
inf-quantavius:retry-root-model-behavior-error

Conversation

@inf-quantavius

@inf-quantavius inf-quantavius commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Problem

The openai-agents SDK raises ModelBehaviorError: Model did not produce a final response! (agents/run_internal/run_loop.py) when a model stream truncates without a terminal event. On the root agent turn this escaped OpenAiAgentRunner's retry loop — is_retriable_llm_error only knew openai/httpx error classes, so the except at openai_agent_runner.py re-raised it, _drive() in engine/main.py called output_bus.fail(exc), and the whole run failed within seconds on a single transient stream truncation.

Failing examples (dev smokes):

  • 2026-06-16 — run 988eaa3d, conversation 520914bc
  • 2026-07-14 — run a91ba571-4fd3-4960-a2a2-de2fbb4dbac8

Subagents only survived this via guarded_invoke's blanket except Exception in subagent_tool_factory.py (#67), which converts the crash into a tool-failure result rather than retrying.

Fix

Classify ModelBehaviorError as retriable in engine/agents/llm_retry.py::is_retriable_llm_error. That's the whole behavioral change — it slots into the runner's existing mid-stream recovery machinery (INF-3504 / INF-3308):

  • rerun from local conversation history, with a trailing incomplete tool turn trimmed (AgentContext.trim_incomplete_tool_turn) so the replayed message array stays valid
  • full-jitter exponential backoff between attempts (retry_backoff_base / llm_retry_backoff_base_seconds, 0 in tests)
  • bounded by the MAX_CONSECUTIVE_LLM_FAILURES = 10 circuit breaker → EngineAgentExhaustedError when exhausted

Retrying is safe here because HALO rebuilds every request from the local AgentContext (no server-side conversation state), which is the same reason non-terminal 400s are already retried. All ModelBehaviorError variants (no final response, malformed tool-call JSON, nonexistent tool) are nondeterministic sampled-output misbehavior, so a re-sampled turn can succeed. Deliberately not retried: MaxTurnsExceeded (must terminate the run), UserError, and guardrail tripwires.

Bonus: subagent turns now retry in-runner before falling back to guarded_invoke's tool-failure result.

Tests

  • tests/unit/agents/test_llm_retry.py: ModelBehaviorError variants are retriable; MaxTurnsExceeded / UserError are not.
  • tests/unit/agents/test_openai_agent_runner.py::test_runner_retries_model_behavior_error_when_stream_has_no_final_response: fake stream yields a partial assistant message then raises the exact production error; asserts the rerun replays local history, the run completes, and the failure counter resets.

Ran locally: pytest tests/unit462 passed; pytest tests/integration -m "not live" → 53 passed (11 Pyodide-sandbox failures are pre-existing on a clean tree on this host — see #67-era note); ruff check / ruff format clean; basedpyright → 0 errors.

🤖 Generated with Claude Code


Note

Medium Risk
Changes when agent runs retry vs fail on root turns; behavior is bounded by the existing circuit breaker and deliberately excludes deterministic SDK errors like max turns.

Overview
Treats the Agents SDK ModelBehaviorError as a retriable LLM failure so truncated streams (e.g. “Model did not produce a final response!”) no longer abort the root agent run.

is_retriable_llm_error now returns true for all ModelBehaviorError variants (missing final response, bad tool JSON, unknown tools), while MaxTurnsExceeded and UserError stay non-retriable. Those errors plug into the existing OpenAiAgentRunner mid-stream recovery: rerun from local AgentContext, trim incomplete tool turns when needed, backoff, and the consecutive-failure circuit breaker.

Tests cover classification and a runner case where a partial assistant message is preserved and the second attempt completes the run.

Reviewed by Cursor Bugbot for commit 2c77514. Bugbot is set up for automated code reviews on this repo. Configure here.

The openai-agents SDK raises ModelBehaviorError ("Model did not produce
a final response!") when a stream ends without a terminal event. On the
root agent turn this escaped OpenAiAgentRunner's retry loop as
non-retriable and failed the whole run; subagents survived only via
guarded_invoke's blanket catch, reporting a tool failure instead of
retrying.

Classify ModelBehaviorError as retriable in is_retriable_llm_error so it
engages the existing mid-stream recovery: trim any incomplete tool turn,
rerun from local conversation history with full-jitter backoff, bounded
by the MAX_CONSECUTIVE_LLM_FAILURES circuit breaker. Other
AgentsException subclasses (MaxTurnsExceeded, UserError, guardrail
tripwires) stay non-retriable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant