Skip to content

responses: fold EasyInputMessage items that omit type - #102

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-08-22_responses-easy-input-normalize
Open

responses: fold EasyInputMessage items that omit type#102
ranxianglei wants to merge 1 commit into
masterfrom
2026-08-22_responses-easy-input-normalize

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Problem

OpenAI's Responses API accepts input message items without an explicit type (the EasyInputMessage shorthand: { role: "user", content: "..." }). responsesToCore dispatched its item switch on item.type, so shorthand items fell through the default branch into the opaque preamble and were never folded:

  • Hosts sending the shorthand for every turn (the omp adapter's buildResponsesInput) had their ENTIRE conversation bypass compression — fold size stayed 0 and compress always rejected with "0 chars".
  • The billion-context proxy serving bare /v1/responses clients (curl etc., all legal shorthand) hit the same hole, surfaced in release v0.1.4 billion-context-omp#12 and the comparison against billion-context.

The omp adapter had to pre-normalize payloads itself (normalizeResponsesPayload); the proxy does not, so the fix belongs in the kernel.

Fix

Normalize type-less items carrying a message role (user/assistant/system/developer) to the canonical typed form before dispatch:

  • user/assistant fold as text messages (string or content-part arrays both handled downstream).
  • system/developer join systemParts exactly like their typed counterparts (hosts re-inject via injectResponsesDeveloperMessage).
  • patchResponsesInput splices folded text back into the slot; re-emitted items carry type: "message" — the shorthand and typed forms are interchangeable on the wire (verified live against SGLang's /v1/responses over a 9-turn tool-loop session).
  • Items with neither type nor a message role keep the opaque preamble path (unchanged).

The caller's body object is never mutated — normalization works on copies.

Tests

4 new cases in tests/wire-bili-message-roundtrip.test.ts:

  1. Shorthand user/assistant items fold like typed messages (preamble stays empty).
  2. patchResponsesInput round-trip: patched text lands in the slot, untouched slots keep content, output in typed form.
  3. Shorthand system role joins systemParts alongside instructions.
  4. Type-less non-message items stay opaque.

Full pre-flight: tsc --noEmit clean, 395 tests pass, build OK.

Downstream

  • billion-context-omp: the adapter-local normalizeResponsesPayload becomes redundant once it pins a kernel release with this fix (kept for now; harmless double-normalize).
  • billion-context (proxy): shorthand bodies served to bare clients become compressible with no proxy-side change.

Refs ranxianglei/billion-context-omp#12

OpenAI's Responses API accepts input message items without an explicit
type (EasyInputMessage shorthand: { role, content }). responsesToCore's
item switch dispatched on item.type, so those items fell through the
default branch into the opaque preamble and were never folded — a
conversation sent entirely in shorthand form (omp host, bare curl
clients) bypassed compression completely.

Normalize type-less items carrying a message role to their canonical
typed form before dispatch: user/assistant fold as text messages,
system/developer join systemParts like their typed counterparts, and
patchResponsesInput re-emits patched slots as typed messages (the two
forms are interchangeable on the wire). Items with neither type nor a
message role keep the opaque preamble path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant