Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions devlog/_plan/260908_a_stack_responses_compat/000_plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# 000 — a_stack_responses_compat: Plan

## Objective

Land four Responses-compatibility changes on `dev` as one manual dependent branch
chain whose tip carries all of them, so a single CI run certifies the whole set.
Three layers carry existing contributor pull requests; one is new work for an
issue that has no pull request.

| Layer | Source | Author to preserve | Subject |
|---|---|---|---|
| 1 | PR #3906, commit `11c498b6c` | MohamadSabree8 | Muse Spark Contributor Free tiers keep unsupported `web_search` fields |
| 2 | PR #3886, commit `83c1d9b12` | cb8010d6 | Spark streams end `adapter_eof` when the Responses Lite header is present |
| 3 | Issue #3922, new work | — | Claude optional tool parameters become strict on Responses routes |
| 4 | PR #3917, commit `2430724e5` | mashfromband | Routed destinations reject Codex `agent_message` with 422 |

Evidence base: four read-only `gpt-6-astra` explorer lanes read the current tree at
`2abf071e0` and returned quoted `path:line` anchors, reproduced in each phase doc.

## Loop-spec

- Loop archetype: satisfy-spec. Each layer has a stated correct behavior; there is
no metric to optimize.
- Trigger: maintainer request to execute workstream A as a stack.
- Goal: the tip merged into `dev`, children settled with authors preserved, linked
issues closed.
- Non-goals: registry `modelWireDefaults` for the `-free` ids; setting the Lite
header to `"false"` instead of removing it; PR #3838's tool-promotion,
`customToolWireName` export and `statelessResponses` work; any other open PR;
`main`/`preview` promotion.
- Verifier: the single Cross-platform CI run on the tip pull request's head SHA.
It runs the repository's own workflow over the cumulative tree, so it observes
every file changed by all four layers.
- Write scope: `src/adapters/openai-responses.ts`, `src/adapters/opencode-go.ts` ->
`src/adapters/routed-agent-messages.ts`, `src/claude/inbound-content-options.ts`,
their regressions, the two test-layout registries,
`docs-site/src/content/docs/reference/adapters.md`,
`docs-site/src/content/docs/reference/configuration/providers.md`, and this unit.
- Budget: no local suite runs at all (instructed). Wall-clock bound is the CI run
plus merge; a red tip after one bounded repair attempt is BLOCKED, not DONE.
- Stop condition: the stack landed on `dev`, proven for the merge method actually
used (see 050): original-tip ancestry for a merge commit, or landed-commit
ancestry plus per-path content equality for squash and rebase.
- Memory artifact: this unit, plus the goalplan at
`.codexclaw/goalplans/deliver-opencodex-workstream-a-responses-compati/`.
- Escalation: a finding that changes a carried author's intended behavior, or a
provider that rejects an explicit `strict: false`, returns to the maintainer.

## Constraints (from the requesting maintainer)

- No local product suite, typecheck, build or install runs in this session. Every
such check is recorded `NOT RUN`.
- Every push uses `--no-verify`.
- CI triggers on the stack tip only. When that one run is green, the tip merges
into `dev`; the remaining pull requests are then settled and the issues closed.
- Carried work keeps its original author through a `Co-authored-by` trailer.

## Why tip-only CI is achievable

`.github/workflows/ci.yml` declares `pull_request: {}` with no base filter, and
`push: branches: [main, preview, dev]`. Pushing `codex/a-stack-l1..l3` starts no
workflow: those refs are not integration branches and no pull request points at
them. Opening exactly one pull request, for layer 4 against `dev`, produces
exactly one Cross-platform CI run whose head contains all four layers.

## Base and chain

Base: `origin/dev` = `942c028735d39b2ad410b1baa95670984e16576d`.

```
codex/a-stack-l4-routed-agentmsg (tip, the only pull request) -> base dev
codex/a-stack-l3-claude-strict
codex/a-stack-l2-spark-lite
codex/a-stack-l1-muse-free
origin/dev 942c02873
```

An ordinary dependent branch chain. GitHub native stacks are not used and were
not requested.

## Work-phase map (one phase = one full PABCD cycle)

| WP | Doc | Slice | Depends on |
|----|-----|-------|------------|
| wp0 | 000 | This roadmap (docs only) | — |
| wp1 | 010 | Layer 1, carry #3906 | wp0 |
| wp2 | 020 | Layer 2, carry #3886 | wp1 |
| wp3 | 030 | Layer 3, implement #3922 | wp2 |
| wp4 | 040 | Layer 4, carry #3917 | wp3 |
| wp5 | 050 | Publish, one CI run, merge, settle | wp4 |

Ordering follows textual adjacency in `src/adapters/openai-responses.ts`, which
layers 1, 2 and 4 all touch at distinct hunks (≈2125, ≈2503, ≈2366 and its import).
Layer 3 touches `src/claude/inbound-content-options.ts` only.

## Terminal outcomes

- DONE — the stack landed on `dev` with the merge-method-specific proof recorded,
children settled with authors preserved, issues #3885/#3922/#3911 closed.
- BLOCKED — CI red on the tip after a bounded repair attempt, or an unmet merge
requirement.
- NEEDS_HUMAN — an audit finding that would change a carried author's intended
behavior beyond what the issue asks.
62 changes: 62 additions & 0 deletions devlog/_plan/260908_a_stack_responses_compat/010_phase1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# 010 — Phase 1: carry PR #3906 (Muse Spark Contributor Free web_search strip)

Branch `codex/a-stack-l1-muse-free`, based on `origin/dev` `942c02873`.
Carried commit `11c498b6c62ae9f7c5b0d25ca24fc6612f607a5c` by MohamadSabree8.

## Problem

`stripMuseSparkUnsupportedWebSearchFields` removes `search_content_types` and
`indexed_web_access` from a Codex `web_search` tool before it reaches the Zen
Responses wire, because that gateway 400s on them. The model guard only lists the
two paid contributor ids, so the two Contributor Free ids ride the same wire and
same gateway contract but keep the rejected fields.

## MODIFY map

`src/adapters/openai-responses.ts` — the constant at 2125-2128.

Before:

```ts
const MUSE_SPARK_WEB_SEARCH_STRICT_MODELS = new Set([
"muse-spark-1.3-contributor",
"muse-spark-1.2-contributor",
]);
```

After:

```ts
const MUSE_SPARK_WEB_SEARCH_STRICT_MODELS = new Set([
"muse-spark-1.3-contributor",
"muse-spark-1.3-contributor-free",
"muse-spark-1.2-contributor",
"muse-spark-1.2-contributor-free",
]);
```

Nothing else changes. The consumer at 2148, its model guard at 2155
(`if (!MUSE_SPARK_WEB_SEARCH_STRICT_MODELS.has(modelId.trim().toLowerCase())) return body;`),
the destination guard at 2159-2164 and the call site at 2451 are untouched.

## TESTS

`tests/providers/muse-spark-web-search-compat.test.ts` — add free-tier cases that
mirror the paid-tier assertions already in the file:

- top-level `tools`: type stays `web_search`, `search_context_size` preserved,
`search_content_types` and `indexed_web_access` absent (mirrors 81-87, 131-137);
- nested `input[].additional_tools.tools`: same removal (mirrors 106-114, 150-158);
- `web_search_preview` untouched for the free ids (mirrors 90-97, 140-147).

## Known limit (recorded, not fixed here)

`src/providers/registry.ts:1685-1690` maps only the paid ids in
`modelWireDefaults`, so the `-free` ids do not select the Responses wire
automatically; this fix applies when that wire is chosen explicitly. Changing the
registry is out of scope, matching the carried pull request.

## Verification (C)

No local command. The layer is verified by the single tip CI run described in 050.
Local suites: NOT RUN by instruction.
73 changes: 73 additions & 0 deletions devlog/_plan/260908_a_stack_responses_compat/020_phase2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# 020 — Phase 2: carry PR #3886 (disable Responses Lite transport for Spark)

Branch `codex/a-stack-l2-spark-lite`, based on layer 1.
Carried commit `83c1d9b129b80d4f65a797fd61a2026deb8c8123` by cb8010d6.

## Problem

Issue #3885: with `x-openai-internal-codex-responses-lite: true`, the canonical
backend opens a `gpt-5.3-codex-spark` SSE response and closes it before a terminal
event, which the adapter reports as `response.incomplete` / `adapter_eof`. The same
request without that header completes.

## MODIFY map

`src/adapters/openai-responses.ts` — inside the canonical-forward block, before
the existing routing-hint work. Line numbers here are against the pinned base
`942c02873` (block at 2503-2513); layer 1 adds two lines above it, so on this
branch the block sits at 2505-2515.

After:

```ts
if (isCanonicalOpenAiForwardProvider(provider)) {
// Spark closes Responses Lite streams before a terminal completion. Select compatibility
// from the final wire model so aliases cannot leave the caller or a static header enabled.
if (isPlainObject(finalBody) && finalBody.model === "gpt-5.3-codex-spark") {
for (const name of Object.keys(headers)) {
if (name.toLowerCase() === CODEX_RESPONSES_LITE_HEADER) delete headers[name];
}
}
const routingHeaders = new Headers(headers);
applyCodexRoutingHint(routingHeaders, finalBody);
```

`finalBody` is computed at 2494-2502 and serialized at 2523 on the pinned base
(2496-2504 and 2525 on this branch), so it is the actual wire model.
`parsed.modelId` can differ; the existing test at 187-188 pins that distinction
deliberately. Keying on `finalBody.model` therefore also covers aliases. The loop
removes every case spelling, which matters because static provider headers merge
in at 2315 and 2353 on the pinned base (2317 and 2355 here) with arbitrary casing.

## Scope of the fix, and what it does not cover (audit finding 1)

The independent audit established a real boundary, verified against source:

- **Covered.** Removing the header fixes the reported defect on the HTTP header
replay path, including the HTTP fallback: `prepareCodexHttpInit()` recomputes
only the routing hint (`src/server/responses/codex-ws-request.ts:46-52`) and
`httpInit` carries the header-deleted request forward (`:68`).
- **Not covered.** On the WebSocket path, `codex-ws-request.ts:30-33` writes
`client_metadata[CODEX_RESPONSES_LITE_METADATA_KEY]` only when the header is
present and reads `"true"`/`"false"`. Deleting the header leaves any
pre-existing `client_metadata` Lite value in the body untouched, and that value
reaches the frame at `:71` and the pool reuse key at
`codex-ws-pool.ts:53-55`.

Setting the header to `"false"` instead of deleting it would also cover the WS
case, but that expands the carried author's diff beyond issue #3885 and changes
socket reuse identity. This phase carries the author's delete-only form and
records the WS metadata case as unresolved rather than silently expanding scope.
The tip pull request states this limit explicitly so the residual is visible.

## TESTS

`tests/codex-integration/codex-metadata-integrity.test.ts`, beside the mixed-case
test at 171: Spark wire model with a caller-provided Lite header, with a
mixed-case static header, and with `parsed.modelId` set to an alias while the
serialized model is Spark; `gpt-5.6-sol` keeps the header. Existing guards at 185,
211, 225 and 267 stay intact.

## Verification (C)

No local command. Verified by the single tip CI run in 050. Local suites: NOT RUN.
141 changes: 141 additions & 0 deletions devlog/_plan/260908_a_stack_responses_compat/030_phase3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# 030 — Phase 3: fix issue #3922 (Claude tool strict default on Responses routes)

Branch `codex/a-stack-l3-claude-strict`, based on layer 2. New work; no existing
pull request. This is the only layer authored here.

## Problem

When Claude Code sends a custom tool without an explicit `strict`, the
Messages -> Responses translation emits a function tool that also omits `strict`.
The Responses API treats an omitted `strict` as an attempt to normalize the schema
into strict mode, so parameters that the Anthropic `input_schema` marks optional
are treated as required upstream, and a tool call that omits them fails. Anthropic
requires an explicit `strict: true` to opt in, so the two defaults disagree.

## MODIFY map

`src/claude/inbound-content-options.ts` — `toolsToResponses`, the function-tool
branch at 26-33.

Before:

```ts
if (typeof raw.name === "string" && raw.name.length > 0 && isRec(raw.input_schema)) {
out.push({
type: "function",
name: raw.name,
...(typeof raw.description === "string" ? { description: raw.description } : {}),
parameters: raw.input_schema as Record<string, unknown>,
});
continue;
}
```

After:

```ts
if (typeof raw.name === "string" && raw.name.length > 0 && isRec(raw.input_schema)) {
out.push({
type: "function",
name: raw.name,
...(typeof raw.description === "string" ? { description: raw.description } : {}),
parameters: raw.input_schema as Record<string, unknown>,
// Anthropic opts into strict tool use explicitly, while Responses normalizes
// an omitted strict into strict mode. Carry the source intent rather than the
// destination default, so an optional input_schema parameter stays optional.
strict: typeof raw.strict === "boolean" ? raw.strict : false,
});
continue;
}
```

The value is derived from the source tool rather than hardcoded, so an explicit
`strict: true` or `strict: false` from the client survives translation, which is
what issue #3922 asks for.

## How the existing strict-tool admission policy relates (audit finding 2)

An earlier draft of this document described `src/claude/compatibility.ts:102`
backwards. What that line actually does:

```ts
if (tool.strict === true) codes.add("strict_tools");
```

`strict_tools` is listed as an incompatible feature at `compatibility.ts:16`, and
`analyzeClaudeCompatibility` (`compatibility.ts:179`) returns `"reject"` for an
incompatible request under enforce mode (decision expression at `:189`), which
`src/server/claude-messages.ts:733-740` applies **before**
translation. So an explicit `strict: true` is already refused in enforce mode and
only reaches translation on the default and shadow paths.

This phase does not change that policy. Detection reads the **source** Anthropic
tool before translation, so emitting a default `strict: false` downstream adds no
new rejection; `tests/claude-integration/claude-compatibility.test.ts:73-78`
already covers the explicit-false allow case, and `:24` lists strict tools among
rejected features.

## Propagation (independently traced twice, no further change needed)

- `src/claude/inbound-content-options.ts:26-32` builds the tool.
- `src/claude/inbound.ts:350-351` assigns it to `body.tools`.
- `src/server/claude-messages.ts:875,897` serializes that body into the internal
Responses request; `src/server/responses/core.ts:3160` parses it.
- `src/responses/schema.ts:114` accepts `strict`; `parser-tools.ts:63` preserves an
explicit value including `false`.
- `src/responses/parser.ts:570` keeps `_rawBody`, and
`src/adapters/openai-responses.ts:2362-2364` starts from it. Canonical-field
stripping only removes `external_web_access`/`defer_loading` (`:189-200`) and
schema normalization spreads the tool (`{ ...tool, parameters }`, `:648-658`),
so `strict` reaches `JSON.stringify(finalBody)` at `:2523`.

## Boundaries

- Hosted `web_search` leaves the function at 22-24, before this branch, so it gains
no `strict` field.
- Native Anthropic passthrough never reaches translation:
`src/server/claude-messages.ts:721-722` returns from `anthropicNativePassthrough`
before the translation call at 757.
- Other Anthropic server tools still drop at 35.

## Schema promise, stated precisely (audit finding 4)

At translation, `parameters` is the caller's `input_schema` reference
(`inbound-content-options.ts:31`), so `properties`, `required` and nested schemas
are unchanged there. That is not a promise of a byte-identical schema on every
outbound route: `openai-responses.ts:651` runs `normalizeXaiToolParameters`, `:657`
supplies a root `type: "object"` when absent, `responses-code-mode.ts:23-27` can
rewrite an `exec` parameter description, and Azure Chat sanitizes at
`openai-chat.ts:1359-1361`. The regression asserts an unchanged schema through the
ordinary OpenAI Responses route.

Adding the field also shifts fallback cache-cohort hashes, because translated tool
definitions participate in the hash at `src/claude/inbound.ts:386-392`. That is a
cohort change, not a correctness change.

## Known risk

The same translated tools feed translated Chat Completions routes, where
`openai-chat.ts:1343` forwards an explicit `strict`; Azure deletes it at `:1364`,
and `tests/providers/azure-model-router-tool-schema.test.ts:42` already pins that
absence. No repository-declared rejection of an explicit `strict: false` was found,
but universal upstream acceptance is not proven. A provider rejecting it is an
escalation.

## TESTS

`tests/claude-integration/claude-inbound.test.ts`:

- **Update the existing assertion at 80-83.** It is an exact `toEqual` on the
translated `Read` tool and will fail once `strict` is present; the expected
object gains `strict: false`.
- Keep `expect(tools[1]).toEqual({ type: "web_search" })` at 84 unchanged.
- New cases: omitted `strict` -> `false`; explicit `false` -> preserved; explicit
`true` -> preserved; an `input_schema` with one required and one optional
property keeps its `required` array through `parseRequest`.
- Assert the three values on the **serialized adapter output**, not only the
translator return, so the wire body is what is pinned.

## Verification (C)

No local command. Verified by the single tip CI run in 050. Local suites: NOT RUN.
Loading
Loading