spec: OpenAI-Compatible Native Tool Return

openspec/changes/openai-compatible-native-tool-return/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-07-06

openspec/changes/openai-compatible-native-tool-return/proposal.md

@@ -0,0 +1,54 @@
+# Proposal: OpenAI-Compatible Native Tool Return
+
+## Summary
+
+Subclass `OpenAIChatModel` in agentpool to override tool return handling for OpenAI-compatible models (e.g., GLM-5, vLLM-hosted models) whose chat templates natively render list-type tool message content. Currently pydantic-ai always serializes list-type tool return values to JSON strings, causing unnecessary escape characters and loss of structural semantics in models that support native list content.
+
+## Motivation
+
+When a tool returns a `list` value (e.g., `["result1", "result2"]`), pydantic-ai's `OpenAIChatModel` serializes it to a JSON string `'["result1", "result2"]'` via `tool_return_ta.dump_json(value).decode()`. This string is then placed as the `content` field of a `ChatCompletionToolMessageParam`.
+
+The OpenAI SDK's `ChatCompletionToolMessageParam.content` type actually supports `str | Iterable[ChatCompletionContentPartTextParam]`, but pydantic-ai only uses the `str` branch.
+
+Models like GLM-5 have chat templates that explicitly branch on `m.content is string` vs list:
+
+```jinja
+{%- if m.content is string -%}
+    {{- '<|tool_return|>' }} {{- m.content }} {{- '<|/tool_return|>' }}
+{%- else -%}
+    {% for tr in m.content %}
+    <|tool_return|>{{ tr.output if tr.output is defined else tr }}<|/tool_return|>
+    {% endfor -%}
+{% endif -%}
+```
+
+When pydantic-ai sends `'["result1", "result2"]'` (a string), the template renders a single `<|tool_return|>` block containing escaped JSON text. If the content were a native list, the template would iterate and create multiple `<|tool_return|>` blocks — the intended behavior for multi-item tool results.
+
+### Problems Caused
+
+1. **Unnecessary escaping** — List data like `[{"name": "Alice"}, {"name": "Bob"}]` becomes `'[{"name": "Alice"}, {"name": "Bob"}]'` — the model sees literal JSON with quotes/brackets as text tokens rather than structured content.
+2. **Chat template mismatch** — Models with list-aware templates (GLM-5, some vLLM deployments) cannot exercise their native multi-block rendering path.
+3. **No escape hatch** — There is no way to configure pydantic-ai to send native list content for OpenAI-compatible endpoints. Issue [#3888](https://github.com/pydantic/pydantic-ai/issues/3888) proposes a `model_response_str` protocol but is still open.
+
+## Proposal
+
+Create an `OpenAICompatibleModel` subclass of `OpenAIChatModel` in agentpool that:
+
+1. **Overrides `_map_user_message()`** to intercept `ToolReturnPart` handling — when the tool return content is a list with no multimodal files, construct `list[ChatCompletionContentPartTextParam]` (i.e., `[{"type": "text", "text": item}, ...]`) instead of a JSON-serialized string.
+2. **Uses a profile flag** (`openai_chat_tool_return_as_list: bool`) to control whether native list content is emitted, defaulting to `False` for backward compatibility.
+3. **Is configurable via manifest YAML** using `ImportModelConfig` or a dedicated config type, allowing agents to opt into this behavior per-model.
+
+### Non-Goals
+
+- Modifying pydantic-ai upstream (this is an agentpool-level override)
+- Supporting the Responses API (which already handles list content for multimodal returns)
+- Changing behavior for non-OpenAI providers (Anthropic, Google, etc.)
+- Auto-detecting whether a model supports list tool content (explicit configuration only)
+
+## References
+
+- pydantic-ai Issue #3888: https://github.com/pydantic/pydantic-ai/issues/3888
+- pydantic-ai Issue #2034: https://github.com/pydantic/pydantic-ai/issues/2034
+- pydantic-ai PR #3826: https://github.com/pydantic/pydantic-ai/pull/3826
+- agentpool Issue #112: https://github.com/Leoyzen/agentpool/issues/112
+- GLM-5 chat template: https://www.modelscope.cn/models/ZhipuAI/GLM-5/resolve/master/chat_template.jinja

openspec/changes/openai-compatible-native-tool-return/specs/openai-compatible-model/spec.md

@@ -0,0 +1,84 @@
+## ADDED Requirements
+
+### Requirement: OpenAICompatibleModel subclass
+The system SHALL provide an `OpenAICompatibleModel` class that subclasses `OpenAIChatModel` from pydantic-ai, located at `agentpool/models/openai_compatible.py`. This class SHALL inherit all functionality from `OpenAIChatModel` and override `_map_user_message()` to optionally emit native list content for tool return messages.
+
+#### Scenario: Class inherits from OpenAIChatModel
+- **WHEN** `OpenAICompatibleModel` is instantiated
+- **THEN** it SHALL be a subclass of `OpenAIChatModel` and accept constructor parameters (`model_name`, `base_url`, `api_key`, `provider`, `tool_return_as_list`, `profile`, `settings`)
+
+#### Scenario: Default behavior matches parent
+- **WHEN** `OpenAICompatibleModel` is instantiated without the `openai_chat_tool_return_as_list` profile flag (or with it set to `False`)
+- **THEN** `_map_user_message()` SHALL delegate entirely to `super()._map_user_message(message)` — producing identical output to `OpenAIChatModel`, with tool return content JSON-serialized to a string
+
+### Requirement: Custom profile TypedDict for type safety
+The system SHALL define an `OpenAICompatibleModelProfile` TypedDict in `agentpool/models/openai_compatible.py` that extends `OpenAIModelProfile` from pydantic-ai with the additional `openai_chat_tool_return_as_list: bool` key. This ensures type-safe access to the profile flag without modifying pydantic-ai upstream.
+
+#### Scenario: Profile flag is type-safe
+- **WHEN** `OpenAICompatibleModel` accesses `self._resolved_profile.get('openai_chat_tool_return_as_list', False)`
+- **THEN** the access SHALL be type-checked by pyright/mypy without errors, because `OpenAICompatibleModelProfile` declares the key and `_resolved_profile` casts to it (following the `OpenRouterModel._resolved_profile` pattern)
+
+### Requirement: Native list tool return when profile flag enabled
+When the `openai_chat_tool_return_as_list` profile flag is `True`, and a `ToolReturnPart` has non-empty list content with no multimodal files, the model SHALL emit `ChatCompletionToolMessageParam.content` as `list[ChatCompletionContentPartTextParam]` (i.e., `[{"type": "text", "text": ...}, ...]`) instead of a JSON-serialized string.
+
+#### Scenario: List content with string items
+- **WHEN** `openai_chat_tool_return_as_list` is `True` and a `ToolReturnPart` has `content=["result1", "result2"]` and no files
+- **THEN** the tool message `content` SHALL be `[{"type": "text", "text": "result1"}, {"type": "text", "text": "result2"}]`
+
+#### Scenario: List content with non-string items
+- **WHEN** `openai_chat_tool_return_as_list` is `True` and a `ToolReturnPart` has `content=[{"key": "value"}, 42]` and no files
+- **THEN** each non-string item SHALL be serialized via `content_items(mode='str')` (which uses `tool_return_ta.dump_json(item).decode()`) and wrapped as `{"type": "text", "text": <serialized>}`
+
+#### Scenario: String content unaffected
+- **WHEN** `openai_chat_tool_return_as_list` is `True` and a `ToolReturnPart` has `content="plain string"` and no files
+- **THEN** the tool message `content` SHALL remain a plain string `"plain string"` (not wrapped in a list)
+
+#### Scenario: Empty list falls back to parent
+- **WHEN** `openai_chat_tool_return_as_list` is `True` and a `ToolReturnPart` has `content=[]` (empty list) and no files
+- **THEN** the tool message `content` SHALL be `''` (empty string), matching parent behavior via `model_response_str_and_user_content()`
+
+#### Scenario: Multimodal content unaffected
+- **WHEN** `openai_chat_tool_return_as_list` is `True` and a `ToolReturnPart` has files (multimodal content)
+- **THEN** the parent's behavior SHALL be used (files extracted to user message, text serialized to string)
+
+#### Scenario: Flag disabled falls back to parent
+- **WHEN** `openai_chat_tool_return_as_list` is `False` (or unset)
+- **THEN** `_map_user_message()` SHALL delegate entirely to `super()._map_user_message(message)` — all `ToolReturnPart` content is JSON-serialized to string
+
+### Requirement: Non-ToolReturnPart messages handled identically to parent
+When the profile flag is `True`, all non-`ToolReturnPart` message parts (`SystemPromptPart`, `UserPromptPart`, `RetryPromptPart`) SHALL be handled identically to the parent `OpenAIChatModel._map_user_message()`, including `file_content` accumulation and the final `UserPromptPart` yield for multimodal files.
+
+#### Scenario: UserPromptPart handling
+- **WHEN** a `ModelRequest` contains a `UserPromptPart` and the flag is `True`
+- **THEN** the part SHALL be mapped identically to the parent's implementation
+
+#### Scenario: SystemPromptPart handling
+- **WHEN** a `ModelRequest` contains a `SystemPromptPart` and the flag is `True`
+- **THEN** the part SHALL be mapped identically to the parent's implementation
+
+#### Scenario: RetryPromptPart handling
+- **WHEN** a `ModelRequest` contains a `RetryPromptPart` and the flag is `True`
+- **THEN** the part SHALL be mapped identically to the parent's implementation (using `part.model_response()`, not affected by the list-content override)
+
+#### Scenario: file_content accumulation preserved
+- **WHEN** the flag is `True` and a `ToolReturnPart` with files is processed
+- **THEN** the `file_content` list SHALL be accumulated across all parts and a final `UserPromptPart` with the file content SHALL be yielded, identical to parent behavior
+
+### Requirement: Manifest configuration via ImportModelConfig with `openai_*` kwarg passthrough
+The system SHALL support configuring `OpenAICompatibleModel` through the existing `ImportModelConfig` mechanism in agent manifests. Because `ImportModelConfig.kw_args` is typed as `dict[str, str]`, the `tool_return_as_list` flag SHALL be passed as a string (`"true"`/`"false"`) and coerced to `bool` by the constructor. The constructor SHALL accept `base_url` and `api_key` as convenience parameters and construct an `OpenAIProvider` internally when `provider` is not explicitly given. Additionally, any `openai_*` prefixed key in `kw_args` SHALL be automatically captured via `**profile_overrides` and merged into the profile dict, with known boolean keys coerced from string to `bool`. Non-`openai_*` keys that are not named constructor params SHALL raise `TypeError`.
+
+#### Scenario: ImportModelConfig with tool_return_as_list flag
+- **WHEN** a manifest defines a model variant with `type: import`, `model: agentpool.models.openai_compatible.OpenAICompatibleModel`, and `kw_args` containing `tool_return_as_list: "true"`, `base_url`, `api_key`, and `model_name`
+- **THEN** the resolved model SHALL be an instance of `OpenAICompatibleModel` with the `openai_chat_tool_return_as_list` profile flag set to `True`
+
+#### Scenario: openai_* profile overrides in kw_args
+- **WHEN** a manifest defines `kw_args` with `openai_system_prompt_role: "developer"` and `openai_supports_strict_tool_definition: "false"`
+- **THEN** the resolved model's profile SHALL contain `openai_system_prompt_role` set to `"developer"` (string) and `openai_supports_strict_tool_definition` set to `False` (bool, coerced from string `"false"`)
+
+#### Scenario: Non-openai_* unknown kwarg raises TypeError
+- **WHEN** a manifest defines `kw_args` with a key that does not start with `openai_` and is not a named constructor parameter (e.g. `foo_bar: "baz"`)
+- **THEN** the constructor SHALL raise `TypeError` (standard Python behavior for unexpected kwargs)
+
+#### Scenario: Model variant referencing import config
+- **WHEN** an agent references a model variant name that resolves to an `ImportModelConfig` for `OpenAICompatibleModel`
+- **THEN** the agent SHALL use the `OpenAICompatibleModel` instance with the configured profile

openspec/changes/openai-compatible-native-tool-return/tasks.md

@@ -0,0 +1,41 @@
+# Tasks: OpenAI-Compatible Native Tool Return
+
+## 1. Create `OpenAICompatibleModel` class and profile
+
+- [ ] 1.1 Create `packages/agentpool/src/agentpool/models/openai_compatible.py`
+- [ ] 1.2 Define `OpenAICompatibleModelProfile(OpenAIModelProfile)` TypedDict with `openai_chat_tool_return_as_list: bool` key
+- [ ] 1.3 Define `OpenAICompatibleModel(OpenAIChatModel)` with `@dataclass(init=False)` and custom `__init__` accepting `model_name`, `base_url`, `api_key`, `provider` (`Provider[AsyncOpenAI] | None`, narrower than parent's string-literal union — documented as intentional for YAML-driven config), `tool_return_as_list` (str|bool, coerced to bool), `profile` (`ModelProfileSpec | None`), `settings`, and `**profile_overrides: str` for arbitrary `openai_*` profile overrides; construct `OpenAIProvider` from `base_url`/`api_key` when `provider` is None; merge `tool_return_as_list` + `openai_*` overrides into profile — handle three cases: `profile is None` → create new dict with `cast(ModelProfileSpec, {...})`; `isinstance(profile, dict)` → spread-merge `{**profile, **overrides}`; callable profile → wrap in lambda that injects overrides post-call
+- [ ] 1.4 Define `_OPENAI_BOOL_PROFILE_KEYS` frozenset and `_coerce_profile_value(key, value)` helper that coerces string → bool for known boolean profile keys, returns string for others
+- [ ] 1.5 Add `_resolved_profile` property returning `cast(OpenAICompatibleModelProfile, self.profile)` (following `OpenRouterModel._resolved_profile` pattern)
+- [ ] 1.6 Override `_map_user_message()`:
+  - [ ] 1.6.1 When flag is `False` (default): delegate entirely to `super()._map_user_message(message)` — zero code duplication
+  - [ ] 1.6.2 When flag is `True`: duplicate the parent's method body, replacing only the `ToolReturnPart` branch:
+    - For `ToolReturnPart` with non-empty list content (`isinstance(part.content, list) and part.content`) and no files (`not part.files`): use `part.content_items(mode='str')` to get serialized items, wrap each string item as `ChatCompletionContentPartTextParam(type='text', text=item)`, yield `ChatCompletionToolMessageParam` with `content=list[...]`
+    - For `ToolReturnPart` with string content, empty list, or files: use parent behavior (`model_response_str_and_user_content()`)
+    - For all other part types (`SystemPromptPart`, `UserPromptPart`, `RetryPromptPart`): identical to parent implementation
+    - Preserve `file_content` accumulation and final `UserPromptPart` yield
+    - End with `assert_never(part)` for exhaustive matching
+- [ ] 1.7 Export `OpenAICompatibleModel` and `OpenAICompatibleModelProfile` from `packages/agentpool/src/agentpool/models/__init__.py`
+
+## 2. Tests
+
+- [ ] 2.1 Create `packages/agentpool/tests/models/test_openai_compatible.py`
+- [ ] 2.2 Unit test: `OpenAICompatibleModel` is a subclass of `OpenAIChatModel`
+- [ ] 2.3 Unit test: Default behavior (flag `False`/unset) delegates to super — tool return content is JSON string
+- [ ] 2.4 Unit test: Flag `True` with list string items → `content` is `list[ChatCompletionContentPartTextParam]`
+- [ ] 2.5 Unit test: Flag `True` with list non-string items → each item JSON-serialized and wrapped in `{"type": "text", "text": ...}`
+- [ ] 2.6 Unit test: Flag `True` with string content → `content` remains plain string (not list-wrapped)
+- [ ] 2.7 Unit test: Flag `True` with empty list `[]` → `content` is `''` (falls back to parent)
+- [ ] 2.8 Unit test: Flag `True` with multimodal content (files) → delegates to parent behavior (file extraction + user message)
+- [ ] 2.9 Unit test: Flag `True` with mixed message (UserPromptPart + ToolReturnPart + RetryPromptPart) → only ToolReturnPart with list content is modified, others match parent output
+- [ ] 2.10 Unit test: `file_content` accumulation preserved when flag is `True` and ToolReturnPart has files
+- [ ] 2.11 Integration test: `ImportModelConfig` resolves `OpenAICompatibleModel` with `tool_return_as_list: "true"` from YAML (string coercion to bool)
+- [ ] 2.12 Integration test: `openai_*` profile overrides in `kw_args` are correctly merged into profile (e.g. `openai_system_prompt_role: "developer"`, `openai_supports_strict_tool_definition: "false"`)
+- [ ] 2.13 Integration test: Non-`openai_*` unknown kwarg raises `TypeError`
+- [ ] 2.14 Integration test: End-to-end agent run with `TestModel` or mock verifying the tool message content shape
+
+## 3. Documentation
+
+- [ ] 3.1 Add module docstring to `openai_compatible.py` explaining purpose and usage
+- [ ] 3.2 Add example YAML manifest snippet in docstring showing `ImportModelConfig` usage with `tool_return_as_list: "true"`, `base_url`, and `api_key`
+- [ ] 3.3 Update `packages/agentpool/src/agentpool/models/__init__.py` `__all__` if applicable

src/agentpool/models/__init__.py

@@ -5,6 +5,10 @@
 from agentpool.models.acp_agents import ACPAgentConfig, ACPAgentConfigTypes, BaseACPAgentConfig
 from agentpool.models.agents import AnyToolConfig, NativeAgentConfig  # noqa: F401
 from agentpool.models.manifest import AgentsManifest, AnyAgentConfig
+from agentpool.models.openai_compatible import (
+    OpenAICompatibleModel,
+    OpenAICompatibleModelProfile,
+)
 from agentpool.models.pending_interaction import PendingPermission, PendingQuestion
 
 
@@ -15,6 +19,8 @@
     "AnyAgentConfig",
     "BaseACPAgentConfig",
     "NativeAgentConfig",
+    "OpenAICompatibleModel",
+    "OpenAICompatibleModelProfile",
     "PendingPermission",
     "PendingQuestion",
 ]