Skip to content

Capture Codex cache writes and price them - #322

Closed
jcsalterego wants to merge 3 commits into
mainfrom
jchen/capture-codex-cache-writes
Closed

Capture Codex cache writes and price them#322
jcsalterego wants to merge 3 commits into
mainfrom
jchen/capture-codex-cache-writes

Conversation

@jcsalterego

@jcsalterego jcsalterego commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What

GPT-5.6 reports cache-write tokens in input_tokens_details, which the Codex parser dropped entirely — those tokens landed in input, so cache-write volume was invisible in the chart.

  • codex/parser.ts — read the cache-read and cache-write counters from both the top-level and nested (input_tokens_details) spellings, subtract writes from input, and record them in cacheWrite5m, so the existing store, rollups, and cache-write chart pick them up with no schema change.
  • Counters are read as the largest candidate, not the first present one. Codex keeps serializing the legacy top-level key as 0 once a value moves into input_tokens_details, so ?? would let that stale zero mask the real nested count — on exactly the transcripts this targets.
  • pricing.ts — gpt-5.6 writes price at 0: the split is for visibility, and no published OpenAI rate charges a write premium. Older Codex tiers mirror their input rate instead, so that if one ever does report writes, moving those tokens out of input can't silently undercount them.
  • Parser version 9 → 10, so existing Codex transcripts reparse and pick up the corrected split.

cacheWrite5m is one of Usage's two Anthropic TTL buckets; a provider-reported write with no TTL uses the first. Documented on the type.

Cost impact

Neutral. Written tokens move from input to cacheWrite5m; for gpt-5.6 they stop being billed (they aren't billed by OpenAI), and for every other Codex tier the two rates are equal.

Testing

  • Codex parser test covers both usage shapes, including the realistic one where the zeroed legacy counter sits beside the nested details.
  • Pricing assertions for the gpt-5.6 (0) and older-tier (input-rate) write costs.
  • Full suite passes (694 tests), plus bun run typecheck.

🤖 Generated with Claude Code

GPT-5.6 reports cache-write tokens in `input_tokens_details`, which the Codex
parser dropped: those tokens landed in `input` and were billed at the uncached
rate. Read the write count from both the nested and top-level shapes, subtract
it from `input`, and record it in `cacheWrite5m` so the existing store, rollups,
and cache-write chart pick it up. Bumps the parser version to 10 to reindex.

Fills in the GPT-5.6 cache-write rates too (1.25x uncached input).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jcsalterego jcsalterego self-assigned this Aug 4, 2026
const cached = numericToken(values.cached_input_tokens ?? inputDetails.cached_tokens);
// GPT-5.6 reports cache writes in input_tokens_details. Newer Codex versions may copy this
// through as a top-level field, so accept both shapes while old transcripts remain valid.
const cacheWrite = numericToken(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache-write extraction is unconditional for every Codex model, but only the three gpt-5.6* families got a non-zero cacheWrite5m price in this PR. Any other Codex model whose transcript carries cache_write_tokens / input_tokens_details.cache_write_tokens (gpt-5.5, gpt-5.4*, gpt-5, codex-mini) now has those tokens moved out of input — where they were billed at the full input rate before this change — into cacheWrite5m, which is priced at 0. Concrete scenario: a gpt-5.5 turn reporting input_tokens=100 / cache_write_tokens=40 previously cost 100 tok x $5/M and now costs 60 tok x $5/M, a silent 40% cost undercount that only appears after the version-10 reparse. Either scope the extraction to the models known to report it, or give the other OpenAI entries a real write price.

const input = numericToken(values.input_tokens);
const cached = numericToken(values.cached_input_tokens);
usage.input = Math.max(input - cached, 0);
const cached = numericToken(values.cached_input_tokens ?? inputDetails.cached_tokens);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ?? chain prefers a present-but-zero top-level field over a non-zero nested one. Codex's own TokenUsage struct always serializes cached_input_tokens (it is 0, not absent, when nothing was cached), so on exactly the "newer Codex also reports input_tokens_details" transcripts this PR is written for, values.cached_input_tokens is 0 and inputDetails.cached_tokens is never consulted — cache reads are recorded as 0 and the tokens land in input at the full rate. The added test only passes because it omits the top-level key entirely, which is not the shape real Codex emits. Same hazard in the cacheWrite chain below. Prefer taking the max of the candidates rather than first-non-nullish.

inputDetails.cache_write_tokens ??
inputDetails.cache_write_input_tokens,
);
usage.input = Math.max(input - cached - cacheWrite, 0);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes the reported cache-write count is a subset of input_tokens (as cached_tokens is), and nothing validates it. If the provider reports writes additively instead, the correct uncached input is input, not input - cacheWrite: a turn with input_tokens=100 / cache_write_tokens=40 yields input=60 + cacheWrite=40 = 100 total instead of 140, and when cached + cacheWrite > input the Math.max(..., 0) clamp swallows the discrepancy entirely with no warning. total_tokens is already parsed a few lines down and would let you detect the additive shape (input + output !== total) instead of guessing; at minimum the assumption deserves a comment plus a consistency check.


const total = numericToken(values.total_tokens);
if (totalTokens(usage) === 0 && total > 0) usage.input = total;
if (totalTokens(usage) === 0 && total > 0) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This subtraction is dead code. The branch only runs when totalTokens(usage) === 0, and totalTokens sums all five buckets (src/types.ts:227), so reaching here implies usage.output, cached, and cacheWrite are all 0 — the new expression always reduces to the old usage.input = total. Worse, it reads as if it handles the case it does not: a transcript with total_tokens and input_tokens_details.cached_tokens but no input_tokens never enters this branch (cacheRead is non-zero), so its input stays 0 and the turn is undercounted. If that is the case you meant to fix, the guard needs to key off input === 0 rather than totalTokens(usage) === 0.

Comment thread src/pricing.ts Outdated
"gpt-5.6-terra": { input: 2, output: 12, cacheRead: 0.2, cacheWrite5m: 0, cacheWrite1h: 0 },
"gpt-5.6-luna": { input: 0.2, output: 1.2, cacheRead: 0.02, cacheWrite5m: 0, cacheWrite1h: 0 },
// OpenAI bills GPT-5.6 cache writes at 1.25x the uncached input rate.
"gpt-5.6": { input: 5, output: 30, cacheRead: 0.5, cacheWrite5m: 6.25, cacheWrite1h: 0 },

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 1.25x write multiplier is Anthropic's published cache model (see the header comment on line 14-16, which explicitly scopes 1.25x/2x to Anthropic and says "OpenAI/Codex and Gemini cached input is represented in the shared cacheRead bucket"). OpenAI's documented prompt caching charges no premium for writing to the cache — writes bill at the standard input rate — which is why every other gpt-* entry here keeps cacheWrite5m: 0. If the 1.25x figure is not backed by a published GPT-5.6 rate, every Codex gpt-5.6 session's cache-write tokens get inflated 25% over the input rate they were charged before this PR, and the three new numbers plus the header comment should be reconciled with a cited source.

jcsalterego and others added 2 commits August 4, 2026 11:00
Codex keeps serializing the legacy top-level counters as 0 once a value moves
into `input_tokens_details`, so `??` picked the stale 0 over the real nested
count -- on exactly the transcripts the previous commit targeted, cache reads
recorded as 0 and those tokens stayed in `input`. Take the largest candidate
instead. The test only passed before because it omitted the top-level key, which
is not the shape Codex writes; it now carries the zeroed legacy counter too.

Also drops the dead subtraction in the `total_tokens` fallback, which was only
reachable with every bucket at 0 and so always reduced to `input = total`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The gpt-5.6 write rates were set to 1.25x input, which is Anthropic's published
multiplier applied to OpenAI without a source. No published OpenAI rate charges
a write premium, so gpt-5.6 writes price at 0: the parser splits them out of
`input` for visibility, not to bill them differently.

Older Codex tiers instead mirror their input rate. They are not known to report
cache writes at all, but if one ever does, the parser would move those tokens
out of `input` and a 0 rate would silently undercount them.

Also refreshes the stale pricing header and CLAUDE.md notes on Codex caching.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jcsalterego jcsalterego closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant