fix(relay): validate semantic upstream responses before success - #1
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a unified “semantic output” validator for multiple upstream protocols (OpenAI Chat/Completions/Responses, Claude, Gemini) and integrates it into both non-stream and SSE relay paths to prevent protocol-shell / empty responses from being treated as successful (and billed), while also tightening retry boundaries once a stream has been committed to the client.
Changes:
- Added
relay/responsevalidatorto consistently validate semantic output, tool-call completeness, filtering/refusal, and terminal events across OpenAI/Claude/Gemini (stream + non-stream). - Updated relay handlers to buffer pre-semantic stream chunks, reject semantically-empty 200 responses, and prevent cross-channel retry after any response bytes are written.
- Added targeted tests for empty shells, tool-call validation/aggregation, terminal rules, and retry-state resets; removed an obsolete poster asset and updated local backup documentation.
Reviewed changes
Copilot reviewed 16 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| relay/responsevalidator/validator.go | New cross-provider semantic/terminal/tool-call validation (stream + non-stream). |
| relay/responsevalidator/validator_test.go | Unit tests for validator edge cases (empty shells, tool-only, truncation, terminal rules). |
| relay/helper/common.go | Adds StreamError helper to emit terminal error after SSE commit. |
| relay/common/relay_info.go | Adds ResetResponseStateForRetry to clear per-attempt response/stream state. |
| relay/common/relay_info_test.go | Tests retry-state reset behavior. |
| relay/channel/openai/relay-openai.go | Applies semantic validation to OpenAI stream + non-stream chat/completions and buffers pre-semantic chunks. |
| relay/channel/openai/relay_responses.go | Applies semantic validation to Responses API (stream + non-stream) and buffers pre-semantic chunks. |
| relay/channel/gemini/relay-gemini.go | Ensures Gemini stream/non-stream responses contain semantic output and enforce terminal presence. |
| relay/channel/gemini/relay-gemini-native.go | Mirrors Gemini semantic/terminal validation for native endpoints (stream + non-stream). |
| relay/channel/claude/relay-claude.go | Applies Claude semantic validation to non-stream + stream (with pre-semantic buffering). |
| relay/channel/claude/relay_claude_tool_use_empty_content_test.go | Updates expectation: tool_use stop reason with empty content is now rejected. |
| output/posters/newapi-40k-stars-light.svg | Removes unused poster asset. |
| MY_COMMITS_BACKUP.md | Updates backup notes for this change set. |
| dto/openai_response.go | Extends OpenAI response DTOs to include fields needed for semantic validation. |
| dto/openai_request.go | Extends Message DTO to carry fields used in response semantic checks. |
| controller/relay.go | Prevents cross-channel retry after response is written; emits in-stream terminal errors when applicable. |
| controller/relay_retry_test.go | Tests shouldRetry rejecting retries once response is committed. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } else if legacy { | ||
| s.Output = true | ||
| } | ||
| for position, call := range delta.ToolCalls { |
Comment on lines
+143
to
+147
| func StreamError(c *gin.Context, format types.RelayFormat, apiErr *types.NewAPIError) { | ||
| if c == nil || apiErr == nil { | ||
| return | ||
| } | ||
| switch format { |
Comment on lines
+272
to
+276
| if strings.TrimSpace(block.Name) == "" { | ||
| return fmt.Errorf("tool_use block has no name") | ||
| } | ||
| index := response.GetIndex() | ||
| s.tools[index] = &streamedToolCall{name: block.Name} |
Comment on lines
+508
to
+512
| tool.name = event.Item.Name | ||
| arguments := event.Item.ArgumentsString() | ||
| if arguments != "" && tool.arguments.Len() == 0 { | ||
| tool.arguments.WriteString(arguments) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 变更描述 / Description
统一校验 OpenAI Chat/Completions/Responses、Claude 与 Gemini 响应是否包含真正可消费的语义输出，避免 role、usage、ping、空 choices/candidates/content 等协议外壳被当作成功并计费。工具调用会聚合并校验名称及 JSON 参数；流式响应在首个语义输出前保留协议前导，已写入客户端后则禁止跨渠道重试，避免拼接不同上游的流。
同时补充重试状态清理、协议内终止错误、边缘场景测试和本地提交备份文档，并按维护需要移除两份旧的 40k stars light 海报。
🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
go test ./... -timeout 60s。📸 运行证明 / Proof of Work
新增测试覆盖空响应外壳、纯工具调用、缺失函数名、工具参数截断、过滤/拒绝、缺失终止事件以及响应已提交后禁止重试。