fix(frontend): return the Anthropic error envelope on bad request bodies

[ayaangazali]

Summary

/v1/messages and /v1/messages/count_tokens answer every failure they produce themselves with the nested Anthropic envelope, and #13102 added the same envelope for unmatched Anthropic routes, with the note that Anthropic clients expect that shape:

{"type":"error","error":{"type":"not_found_error","message":"Route not found: GET /v1/messages/missing"}}

Requests that fail before reaching the handler did not get it. Both handlers used Axum's Json extractor, whose rejections are text/plain. Measured on main with the Anthropic endpoints enabled:

Content-Type: text/plain  ->  415  text/plain  Expected request with `Content-Type: application/json`
body "{not json"          ->  400  text/plain  Failed to parse the request body as JSON: ...

So a client written against the documented envelope parses an unmatched route fine and throws on the two most common request mistakes.

I checked the obvious counter-hypothesis before concluding anything: this router layers anthropic_error_middleware, so it looked like this might already be handled. It is not. That middleware only rewrites 422 to 400, and neither rejection is a 422.

The fix reads the body explicitly and reports unsupported media type, oversized body and malformed JSON through anthropic_error. handler_count_tokens gains the HeaderMap it needs to see the content type.

Content-type parsing reuses is_json_content_type from the openai module rather than reimplementing it, so this route keeps the same notion of what counts as JSON, including parameters and +json suffixes. The policy is unchanged, only the envelope: an absent Content-Type is still rejected, matching ensure_json_content_type today.

Overlap worth flagging

Two things this touches are also touched elsewhere, so whoever merges second should expect a trivial rebase rather than a surprise:

• fix(frontend): return the generate route's own error envelope on bad input #13696 makes the identical one-line change exposing is_json_content_type as pub(super), for the same reason on /inference/v1/generate. Whichever lands first, the other drops that hunk.
• @chanh has fix(frontend): treat an absent Content-Type as JSON #13268 open to treat an absent Content-Type as JSON. If that lands, this route should follow it, and reusing the shared helper is what keeps that a one-place change.

This is the same class as #13685 and #13696 but a third module with a third envelope, so it is deliberately not a copy of either: Anthropic nests under {"type":"error","error":{...}}, generate uses {"error":{...}}, and the openai routes use a flat {message,type,code}.

Validation

Measured before and after by driving the route through spawn_service(|b| b.enable_anthropic_endpoints(true)):

input	before	after
Content-Type: text/plain	415 text/plain	415 application/json, {"type":"error","error":{"type":"invalid_request_error",...}}
{not json	400 text/plain	400 application/json, same envelope

Added test_anthropic_bad_content_type_returns_anthropic_envelope and test_anthropic_malformed_json_returns_anthropic_envelope, asserting the response content-type as well as the body, because the status codes were already correct on main and only the shape was wrong. Both confirmed red first by restoring anthropic.rs from upstream/main and re-running:

the 415 must use the Anthropic envelope, not Axum's text/plain rejection
the 400 must use the Anthropic envelope, not Axum's text/plain rejection

• Full crate: 2043 tests, which is main's 2041 plus the two added, with cargo clippy and cargo fmt --all -- --check clean.
• The one failure in those runs is test_oversized_body_returns_json_413, the pre-existing intermittent I reported on fix(frontend): preserve chat_template_args through Responses conversion #13624. It is unrelated to this change and fails at the same rate on main.

Not verified here: no GPU, so the success path of /v1/messages was not exercised end to end. This change only affects how the request body is read and how that read is reported when it fails.

---

Summary by CodeRabbit

• Bug Fixes
  • Anthropic Messages and token-count requests now provide consistent JSON-formatted error responses for unsupported content types, oversized request bodies, read failures, and malformed JSON.
  • Invalid requests now return the appropriate HTTP status codes, including 400 for malformed JSON and 415 for unsupported media types.
  • Request body size limits are enforced consistently across supported Anthropic endpoints.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

👋 Hi ayaangazali! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 851a9d9b-9564-4a74-a620-ae7b03b88a17

📥 Commits

Reviewing files that changed from the base of the PR and between 004cd02 and 16da40f.

📒 Files selected for processing (3)

• lib/llm/src/http/service/anthropic.rs
• lib/llm/src/http/service/openai.rs
• lib/llm/src/http/service/service_v2.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

Anthropic message and token-count handlers now use centralized request parsing. The parser validates JSON content types, enforces body limits, parses JSON, and returns Anthropic-formatted errors. Integration tests cover unsupported media types and malformed JSON.

Changes

Anthropic request handling

Layer / File(s)	Summary
Shared request parser lib/llm/src/http/service/anthropic.rs, lib/llm/src/http/service/openai.rs	The shared parser validates application/json, enforces the body-size limit, maps failures to HTTP statuses, and formats Anthropic error envelopes.
Handler integration lib/llm/src/http/service/anthropic.rs	The messages and token-count handlers now parse headers and raw bodies through the shared Anthropic parser.
Integration validation lib/llm/src/http/service/service_v2.rs	Tests verify JSON Anthropic error responses for unsupported content types and malformed JSON.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 16da4

This change makes malformed or unsupported Anthropic request bodies return the expected JSON error envelope without changing the existing status-code policy. No actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description explains the implementation and validation, but it omits the required template sections and the required Related Issues confirmation or issue link.	Add the Overview, Details, Where should the reviewer start?, and Related Issues sections, including an issue link or confirmation that no issue exists.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring check was indeterminate for this PR — some files could not be analyzed in time. Not blocking.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: Anthropic error envelopes for invalid request bodies.

---

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.