Support response_format in chat completions#538
Conversation
kolsha
requested review from
LuckyIYI,
alexxale,
eugenebokhan and
uuuvn
as code owners
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c1e629980b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
eugenebokhan
left a comment
eugenebokhan
left a comment
There was a problem hiding this comment.
Could you please update your branch given the latest main commit e3c9a3c fixing the json schema parsing fix?
Enhance chat session and grammar handling with new token IDs - Added `trigger_token_id` parameter to `create_compiled_grammar` for improved grammar configuration. - Introduced `end_of_thinking_token_id` in `ChatSession` to manage end-of-thinking scenarios. - Updated `to_chat_messages` function to include reasoning effort in the last message. - Added new tests for structured output with reasoning enabled and disabled. - Refactored response handling in `ChatCompletionRequest` to support new response formats and reasoning effort.
…d in `StructuredOutput`
- Updated `create_compiled_grammar` to accept multiple trigger token IDs and additional parameters for pre-engagement token management. - Modified `ChatSession` to handle a vector of end-of-thinking token IDs, enhancing flexibility in grammar engagement. - Introduced logic for managing pre-engagement tokens and trimming slack tokens in `CompiledGrammarEngagementState`. - Added tests to validate the new engagement logic and token handling.
kolsha
force-pushed
the
support-response-format
branch
from
c1e6299 to
05a6779
Compare
Sure, if understand correctly - you want the schema parsing logic to live inside backend. I also brought support for multiple trigger tokens. I'm lacking some context, so this might need some polishing. psss |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
This comment was marked as spam.
This comment was marked as spam.
|
Please resolve the conflicts and make sure CI tests pass |
| } | ||
| } | ||
|
|
||
| #[cfg(test)] |
There was a problem hiding this comment.
Please either remove these tests or move them to a dedicated uzu/tests/unit dir
|
Please update to the latest main given an openai response format pr was merged |
…rmat # Conflicts: # crates/cli/src/server/chat_completions.rs
|
Done, please take a look when you have time |
3 participants
Hey, folks!
I've started using uzu in my personal projects and noticed chat completions doesn't support OpenAI-style response_format yet.
I needed it, so I implemented it on top of the existing grammar support and opened a PR.
Happy to tweak the API or approach 🐼
Thanks!
Enhance chat session and grammar handling with new token IDs
trigger_token_idparameter tocreate_compiled_grammarfor improved grammar configuration.end_of_thinking_token_idinChatSessionto manage end-of-thinking scenarios.to_chat_messagesfunction to include reasoning effort in the last message.ChatCompletionRequestto support new response formats and reasoning effort.