feat: Add buffered Chat Completions tool-call streaming#3506
feat: Add buffered Chat Completions tool-call streaming#3506incoffeemonster wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 09033e9b6c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Delightful! ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
Summary
Some OpenAI-compatible Chat Completions providers can return valid function tool calls in the final assembled stream, but their streamed
tool_callschunks may not be reliable enough for the SDK to process incrementally. This can cause the Agents SDK to emit partial tool-call events too early or build invalid assistant/tool message ordering for later turns.This PR adds an opt-in buffered streaming mode for those providers. When
buffer_streamed_tool_calls=Trueis enabled,OpenAIChatCompletionsModelcontinues to stream normal text deltas immediately, but buffers function tool-call deltas until the provider stream completes. It then emits one complete synthetic tool-call chunk into the existingChatCmplStreamHandler, so downstream Responses-format events and session history remain protocol-safe.Default behavior is unchanged.
Key changes:
buffer_streamed_tool_callstoOpenAIChatCompletionsModelandOpenAIProvider.openai_buffer_streamed_tool_callspassthrough toMultiProvider.ModelBehaviorErrorwhen a stream finishes withfinish_reason="tool_calls"but no tool-call deltas were observed.Test plan
Automated checks run locally:
Result:
27 passed.Manual smoke test:
test.py.agentsimports from localsrc/agents.OpenAIChatCompletionsModel(..., buffer_streamed_tool_calls=True).tool_calls/tool_call_idprotocol errors.SQLiteSessioncompleted without stale assistant/tool message ordering errors.Issue number
N/A
Checks
make lintandmake format