This was generated by AI during triage.
Area
Dashboard
What are you trying to accomplish?
Compare routed model generation throughput without conflating first-token latency with post-first-output delivery speed. The Logs page already records TTFT and output token counts, so users should be able to inspect an estimated decode-rate metric alongside the existing end-to-end tok/s value.
What prevents this today?
The current tok/s metric is intentionally calculated as outputTokens / full request duration. A request with a large TTFT therefore appears slow even when tokens arrive quickly after the first output. TTFT is shown separately in the detail dialog, but the dashboard does not derive or display the post-TTFT rate.
What should OpenCodex do?
Keep the existing end-to-end tok/s metric and its filter semantics unchanged. Add an explicitly estimated decode-rate value calculated as:
outputTokens / (durationMs - firstOutputMs)
Use the request-relative TTFT for the top-level row and each attempt's own attempt-relative TTFT for combo rows. Return the metric as unavailable when usage, positive output tokens, TTFT, or a positive post-TTFT duration is unavailable. Label the value as estimated because completion framing, network pauses, and stream finalization can still be included after the first output.
Example usage or interface
For a log entry with 240 output tokens, 10,000 ms total duration, and 2,000 ms TTFT:
- Existing end-to-end rate:
24 tok/s
- Estimated decode rate:
30 tok/s
The Logs table and request detail should show both values without changing existing speed filters.
Alternatives or workarounds
Users can manually export the log fields and compute the value, but this is slow and error-prone, especially for combo attempts whose TTFT is relative to the attempt rather than the parent request.
Additional context
This is a display-time derived metric only. It should not mutate persisted request logs or usage history.
Checks
Proposed acceptance criteria
- Keep the current end-to-end
tok/s value and speed-filter behavior unchanged.
- Add
displayMetrics.decodeTokPerSecond only at management-API response time; do not change RequestLogEntry, usage.jsonl, or persisted schemas.
- Use the parent request's
firstOutputMs for the parent metric and each combo attempt's own firstOutputMs for the attempt metric.
- Return
ttft_missing when TTFT was not recorded, and invalid_duration when the post-TTFT interval is non-positive or invalid.
- Mark every decode-rate value as estimated, including when token usage itself was reported exactly.
- Stack end-to-end and decode values in the existing Logs rate column to avoid widening the fixed-layout table; show separate labeled values in request details and combo attempt details.
- Keep this metric Logs-only; routing analytics remain unchanged.
- Cover parent/attempt separation, unavailable cases, persistence immutability, locale parity, and unchanged speed-filter semantics with tests.
Area
Dashboard
What are you trying to accomplish?
Compare routed model generation throughput without conflating first-token latency with post-first-output delivery speed. The Logs page already records TTFT and output token counts, so users should be able to inspect an estimated decode-rate metric alongside the existing end-to-end
tok/svalue.What prevents this today?
The current
tok/smetric is intentionally calculated asoutputTokens / full request duration. A request with a large TTFT therefore appears slow even when tokens arrive quickly after the first output. TTFT is shown separately in the detail dialog, but the dashboard does not derive or display the post-TTFT rate.What should OpenCodex do?
Keep the existing end-to-end
tok/smetric and its filter semantics unchanged. Add an explicitly estimated decode-rate value calculated as:outputTokens / (durationMs - firstOutputMs)Use the request-relative TTFT for the top-level row and each attempt's own attempt-relative TTFT for combo rows. Return the metric as unavailable when usage, positive output tokens, TTFT, or a positive post-TTFT duration is unavailable. Label the value as estimated because completion framing, network pauses, and stream finalization can still be included after the first output.
Example usage or interface
For a log entry with 240 output tokens, 10,000 ms total duration, and 2,000 ms TTFT:
24 tok/s30 tok/sThe Logs table and request detail should show both values without changing existing speed filters.
Alternatives or workarounds
Users can manually export the log fields and compute the value, but this is slow and error-prone, especially for combo attempts whose TTFT is relative to the attempt rather than the parent request.
Additional context
This is a display-time derived metric only. It should not mutate persisted request logs or usage history.
Checks
Proposed acceptance criteria
tok/svalue and speed-filter behavior unchanged.displayMetrics.decodeTokPerSecondonly at management-API response time; do not changeRequestLogEntry,usage.jsonl, or persisted schemas.firstOutputMsfor the parent metric and each combo attempt's ownfirstOutputMsfor the attempt metric.ttft_missingwhen TTFT was not recorded, andinvalid_durationwhen the post-TTFT interval is non-positive or invalid.