Skip to content

Remove unnecessary json decode-encode#65

Merged
odp merged 2 commits into
prodfrom
duplicate-json-decode
Jul 2, 2026
Merged

Remove unnecessary json decode-encode#65
odp merged 2 commits into
prodfrom
duplicate-json-decode

Conversation

@odp

@odp odp commented Jul 2, 2026

Copy link
Copy Markdown

The outer chat_completion code already decodes result["response_body"]

    @app.post("/sessions/{session_id}/v1/chat/completions")
    async def chat_completions(request: Request, session_id: str):

            response = orjson.loads(result["response_body"])
            ...
            return backend.build_proxy_response(result)

then we do decode and re-encode the bytes again inside build_proxy_response. This is in the hot-path.

also move, json parsing to_thread, can block the event loop.

@odp odp requested a review from a team July 2, 2026 07:07
flukeskywalker
flukeskywalker previously approved these changes Jul 2, 2026

@flukeskywalker flukeskywalker left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a clear win

@odp odp merged commit 701c556 into prod Jul 2, 2026
4 checks passed
@odp odp deleted the duplicate-json-decode branch July 2, 2026 07:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants