feat(models): a caller can ask the provider how big a prompt is

[rezaho]

What

Adds a provider-counted token capability to the model port:

• BaseAPIModel.acount_tokens(messages, *, tools=None, system=None) -> int | None, defaulting to None and delegating to the async adapter when it implements the method.
• AsyncAnthropicAdapter.acount_tokens and AsyncAnthropicOAuthAdapter.acount_tokens, both against POST /v1/messages/count_tokens.

Why

Nothing in the framework could learn a token count; the only counter present is a chars/4 heuristic (marsys/utils/tokens.py). Characters per token is a property of the content, not the model — the same day's traffic on one conversation measured 1.14 to 3.16 chars/token — so any bound or residual computed by subtracting a character estimate from a provider-reported count is unrelated to the quantity it names. The endpoint answers with the model's own tokenizer, is free, is rate-limited independently of message creation, and does not participate in prompt caching, so a caller can size the exact payload it is about to send without perturbing the request it is sizing.

Shape

Each Anthropic leg builds the counted body through its own format_request_payload, then drops only the keys the count endpoint rejects (max_tokens, stream, sampling params, output_config). That is deliberate: the OAuth leg prepends a required Claude-Code system block and renames reserved tool names, so a body assembled any other way would count a request that is never sent. The OAuth leg also refreshes its token first, like its messages call, and asks for application/json instead of the streaming accept.

The two legs are siblings with no shared request path, so the transport code is per-leg (aiohttp for the api-key leg, httpx for the OAuth leg, matching each one's messages call); the URL derivation, the payload stripping and the response interpretation are shared module-level helpers in anthropic.py.

Failure behaviour

None, never an exception — the caller is sizing something, not producing an answer, and a count that fails must cost the request nothing. A 404/405 means the route is absent on that endpoint and is remembered for the process; 401/403 and transport errors are not remembered, because the OAuth credentials file has several writers (sibling adapters, the Claude CLI) and a refresh in flight looks exactly like a rejection for one request — memoising that would disable counting for the lifetime of a long-running process.

Tests

tests/models/test_count_tokens.py, no network: URL derivation (including the OAuth leg's preserved query string), the stripped/kept payload keys, both legs' counted body (the OAuth system prefix and tool renaming are asserted present), None on transport failure / non-JSON body / a 200 without input_tokens, the 404-remembered vs 401-not-remembered split, per-endpoint scoping of the memo, and the model-port delegation including the "provider cannot count" default.

packages/framework/tests/models: 282 passed, 39 skipped.

What a live run added

Driving the endpoint for real, rather than only through fakes, turned up a rule that
binds counts constantly and the messages call almost never:

invalid_request_error: messages: final assistant content cannot end with trailing whitespace

Counting a conversation means presenting its last row as final; a settled
conversation ends with an assistant reply; models end replies with a newline. So the
first caller of this endpoint would have found its counts failing on exactly the
conversations it most wanted to size, with a bare "HTTP 400" and nothing else. The
count payload now right-trims the final assistant message's last text block (copying
rather than mutating — those blocks can be the caller's own durable rows) and drops
the message when nothing is left of it. A non-200 also logs the provider's own
sentence now, because a status alone cost a round trip to diagnose.

Note on the base branch

Opened against azure-openai-leg rather than main, because that is the commit the consuming submodule pin currently points at and both branches touch models/models.py and the two Anthropic adapters. If azure-openai-leg merges first, GitHub retargets this to main.