Retry ModelBehaviorError so a truncated stream no longer kills the run#76
Open
inf-quantavius wants to merge 1 commit into
Open
Retry ModelBehaviorError so a truncated stream no longer kills the run#76inf-quantavius wants to merge 1 commit into
inf-quantavius wants to merge 1 commit into
Conversation
The openai-agents SDK raises ModelBehaviorError ("Model did not produce
a final response!") when a stream ends without a terminal event. On the
root agent turn this escaped OpenAiAgentRunner's retry loop as
non-retriable and failed the whole run; subagents survived only via
guarded_invoke's blanket catch, reporting a tool failure instead of
retrying.
Classify ModelBehaviorError as retriable in is_retriable_llm_error so it
engages the existing mid-stream recovery: trim any incomplete tool turn,
rerun from local conversation history with full-jitter backoff, bounded
by the MAX_CONSECUTIVE_LLM_FAILURES circuit breaker. Other
AgentsException subclasses (MaxTurnsExceeded, UserError, guardrail
tripwires) stay non-retriable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The openai-agents SDK raises
ModelBehaviorError: Model did not produce a final response!(agents/run_internal/run_loop.py) when a model stream truncates without a terminal event. On the root agent turn this escapedOpenAiAgentRunner's retry loop —is_retriable_llm_erroronly knew openai/httpx error classes, so theexceptatopenai_agent_runner.pyre-raised it,_drive()inengine/main.pycalledoutput_bus.fail(exc), and the whole run failed within seconds on a single transient stream truncation.Failing examples (dev smokes):
988eaa3d, conversation520914bca91ba571-4fd3-4960-a2a2-de2fbb4dbac8Subagents only survived this via
guarded_invoke's blanketexcept Exceptioninsubagent_tool_factory.py(#67), which converts the crash into a tool-failure result rather than retrying.Fix
Classify
ModelBehaviorErroras retriable inengine/agents/llm_retry.py::is_retriable_llm_error. That's the whole behavioral change — it slots into the runner's existing mid-stream recovery machinery (INF-3504 / INF-3308):AgentContext.trim_incomplete_tool_turn) so the replayed message array stays validretry_backoff_base/llm_retry_backoff_base_seconds, 0 in tests)MAX_CONSECUTIVE_LLM_FAILURES = 10circuit breaker →EngineAgentExhaustedErrorwhen exhaustedRetrying is safe here because HALO rebuilds every request from the local
AgentContext(no server-side conversation state), which is the same reason non-terminal 400s are already retried. AllModelBehaviorErrorvariants (no final response, malformed tool-call JSON, nonexistent tool) are nondeterministic sampled-output misbehavior, so a re-sampled turn can succeed. Deliberately not retried:MaxTurnsExceeded(must terminate the run),UserError, and guardrail tripwires.Bonus: subagent turns now retry in-runner before falling back to
guarded_invoke's tool-failure result.Tests
tests/unit/agents/test_llm_retry.py:ModelBehaviorErrorvariants are retriable;MaxTurnsExceeded/UserErrorare not.tests/unit/agents/test_openai_agent_runner.py::test_runner_retries_model_behavior_error_when_stream_has_no_final_response: fake stream yields a partial assistant message then raises the exact production error; asserts the rerun replays local history, the run completes, and the failure counter resets.Ran locally:
pytest tests/unit→ 462 passed;pytest tests/integration -m "not live"→ 53 passed (11 Pyodide-sandbox failures are pre-existing on a clean tree on this host — see #67-era note);ruff check/ruff formatclean;basedpyright→ 0 errors.🤖 Generated with Claude Code
Note
Medium Risk
Changes when agent runs retry vs fail on root turns; behavior is bounded by the existing circuit breaker and deliberately excludes deterministic SDK errors like max turns.
Overview
Treats the Agents SDK
ModelBehaviorErroras a retriable LLM failure so truncated streams (e.g. “Model did not produce a final response!”) no longer abort the root agent run.is_retriable_llm_errornow returns true for allModelBehaviorErrorvariants (missing final response, bad tool JSON, unknown tools), whileMaxTurnsExceededandUserErrorstay non-retriable. Those errors plug into the existingOpenAiAgentRunnermid-stream recovery: rerun from localAgentContext, trim incomplete tool turns when needed, backoff, and the consecutive-failure circuit breaker.Tests cover classification and a runner case where a partial assistant message is preserved and the second attempt completes the run.
Reviewed by Cursor Bugbot for commit 2c77514. Bugbot is set up for automated code reviews on this repo. Configure here.