Skip to content

Show reasoning/thinking output for OpenAI Responses API (o-series models) #773

Description

@micz

Show reasoning/thinking output for OpenAI Responses API (o-series models)

Context

When using the OpenAI Responses API with reasoning models (o3, o4-mini, o3-mini, etc.), the
API streams reasoning summary events alongside the normal text output. ThunderAI currently
ignores these events entirely.

This issue tracks adding support for capturing and displaying the reasoning content in the
chat window, collapsed by default (similar to how <think> blocks from Ollama/OpenAI
Compatible models are handled — see #636).


How the reasoning arrives in the stream

The OpenAI Responses API sends SSE events of the following types for reasoning content:

{ "type": "response.reasoning_summary_part.added", "item_id": "...", "output_index": 0, "content_index": 0, "part": { "type": "reasoning_summary_text", "text": "" } }
{ "type": "response.reasoning_summary_text.delta", "item_id": "...", "output_index": 0, "content_index": 0, "delta": "...reasoning chunk..." }
{ "type": "response.reasoning_summary_text.done", "item_id": "...", "output_index": 0, "content_index": 0, "text": "...full reasoning summary..." }

The normal text output uses response.output_text.delta, which is already handled.


Current behaviour

In js/workers/model-worker-openai_responses.js, the worker only handles:

  • response.created — to capture previous_response_id
  • response.output_text.delta — to stream the visible text
  • response.failed — for errors

All response.reasoning_summary_* events fall through silently and are discarded.


Required changes

1. js/workers/model-worker-openai_responses.js

Add a thinkingAccumulator variable and handle the new event type:

let thinkingAccumulator = '';

// inside the parsedLines loop:
} else if (parsedLine.type === 'response.reasoning_summary_text.delta' && parsedLine.delta) {
    thinkingAccumulator += parsedLine.delta;
    postMessage({ type: 'newThinkingToken', payload: { token: parsedLine.delta } });
}

On tokensDone, pass the accumulated thinking to the UI:

postMessage({ type: 'tokensDone', payload: { thinking: thinkingAccumulator } });
thinkingAccumulator = '';

2. api_webchat/messagesArea.js

Handle newThinkingToken alongside newToken, and on tokensDone prepend the thinking
block (as a collapsed <details> element) before rendering the markdown of the main response.

3. Settings/preferences

A global toggle (already planned in #636) to completely hide thinking blocks should cover
this case as well — no separate preference needed.


Important constraints

  • reasoning_summary events are only emitted when store: true is set in the request
    body. Without it, the API does not return a reasoning summary even for o-series models.
    This needs to be documented clearly in the UI.
  • The content received is a summary of the reasoning, not the raw chain-of-thought.
    This is an OpenAI API limitation and cannot be worked around client-side.
  • Not all models support reasoning: only o1, o3, o3-mini, o4-mini and future o-series models.
    For gpt-* models this feature is a no-op.
  • Relevant OpenAI documentation:
    Reasoning models — streaming
    Responses API streaming events reference

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions