How to stream tool call argument deltas in real-time during LLM streaming?

[Lucas-Yu-Chen]

Question

When an LLM generates a tool call with large arguments (e.g., a Python code block) in streaming mode, Spring AI currently merges all argument deltas into a single complete ToolCall object before making it available. There is no way to receive incremental argument fragments in real-time,
which is critical for use cases where the tool call argument itself is the primary output (e.g., code generation tools, artifact generation).

Current Behavior

The streaming pipeline merges tool call argument deltas at multiple layers:

1. Provider-level: OpenAiStreamFunctionCallingHelper (and similar helpers for other providers) accumulates argument fragments via StringBuilder.append() and only emits a complete ToolCall.

2. Framework-level: MessageAggregator collects all ToolCall objects via addAll() and builds a single aggregated AssistantMessage on stream completion.

The net result: when the LLM is generating a long code block inside a tool call argument, the frontend receives nothing until the entire argument is complete — even though the underlying SSE stream delivers incremental tokens.

Example SSE chunks from the API:
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"call_abc","function":{"name":"write_code","arguments":""}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"def "}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"hello"}}]}}]}
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"():\n "}}]}}]}
...
data: {"choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":" print('Hello')"}}]}}]}
data: {"choices":[{"delta":{},"finish_reason":"tool_calls"}]}

Currently, the application only receives the fully merged argument string after [DONE], losing the real-time UX.

Expected Behavior

Provide a callback/handler mechanism that fires for each tool call argument delta, allowing applications to stream tool call arguments (e.g., code) to the frontend in real-time, token by token.

Is there an existing way to achieve this in Spring AI that I might have missed? Or is this a gap that would need a new feature?

[i-zoufeng]

i can fix this