responses: fold EasyInputMessage items that omit type - #102
Open
ranxianglei wants to merge 1 commit into
Open
Conversation
OpenAI's Responses API accepts input message items without an explicit
type (EasyInputMessage shorthand: { role, content }). responsesToCore's
item switch dispatched on item.type, so those items fell through the
default branch into the opaque preamble and were never folded — a
conversation sent entirely in shorthand form (omp host, bare curl
clients) bypassed compression completely.
Normalize type-less items carrying a message role to their canonical
typed form before dispatch: user/assistant fold as text messages,
system/developer join systemParts like their typed counterparts, and
patchResponsesInput re-emits patched slots as typed messages (the two
forms are interchangeable on the wire). Items with neither type nor a
message role keep the opaque preamble path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
OpenAI's Responses API accepts input message items without an explicit
type(the EasyInputMessage shorthand:{ role: "user", content: "..." }).responsesToCoredispatched its item switch onitem.type, so shorthand items fell through thedefaultbranch into the opaque preamble and were never folded:buildResponsesInput) had their ENTIRE conversation bypass compression — fold size stayed 0 andcompressalways rejected with "0 chars"./v1/responsesclients (curl etc., all legal shorthand) hit the same hole, surfaced in release v0.1.4 billion-context-omp#12 and the comparison against billion-context.The omp adapter had to pre-normalize payloads itself (
normalizeResponsesPayload); the proxy does not, so the fix belongs in the kernel.Fix
Normalize type-less items carrying a message role (
user/assistant/system/developer) to the canonical typed form before dispatch:user/assistantfold as text messages (string or content-part arrays both handled downstream).system/developerjoinsystemPartsexactly like their typed counterparts (hosts re-inject viainjectResponsesDeveloperMessage).patchResponsesInputsplices folded text back into the slot; re-emitted items carrytype: "message"— the shorthand and typed forms are interchangeable on the wire (verified live against SGLang's/v1/responsesover a 9-turn tool-loop session).typenor a messagerolekeep the opaque preamble path (unchanged).The caller's body object is never mutated — normalization works on copies.
Tests
4 new cases in
tests/wire-bili-message-roundtrip.test.ts:patchResponsesInputround-trip: patched text lands in the slot, untouched slots keep content, output in typed form.systemrole joinssystemPartsalongsideinstructions.Full pre-flight:
tsc --noEmitclean, 395 tests pass, build OK.Downstream
billion-context-omp: the adapter-localnormalizeResponsesPayloadbecomes redundant once it pins a kernel release with this fix (kept for now; harmless double-normalize).billion-context(proxy): shorthand bodies served to bare clients become compressible with no proxy-side change.Refs ranxianglei/billion-context-omp#12