Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/project/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ Directional roadmap for AgentMonitor. This is a planning snapshot, not a release

Concise record of shipped work that has left `BACKLOG.md`. Newest first.

- Cache-inclusive unknown-pricing visibility (2026-08-04) — *What:* Top Models now
offers an All tokens view that includes input, output, cache-read, and cache-write
traffic; model tables use the same total. The Usage page persistently identifies
pricing-incomplete models, their cache-inclusive observed tokens, and their event count
without fabricating a cost. The warning stays visible when a formerly unknown model
receives pricing but its historical $0 rows still need `amon costs recalc`. *Why:* a
cache-heavy new model can otherwise vanish from the default Cost view before or after
its pricing record arrives.
- Configurable Claude history root (2026-07-29) — *What:*
`AGENTMONITOR_CLAUDE_DIR` now supplies one Claude data root to startup sync,
live and periodic watcher discovery, automatic and historical event import,
Expand Down
1 change: 1 addition & 0 deletions docs/system/FEATURES.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ Product-surface reference for AgentMonitor.
- Summary totals, daily series, project/model/tier/agent attribution, and top-session views all use cost/token-bearing event rows as their source of truth.
- Codex aggregate usage reconciles overlapping telemetry sources: when a Codex session has imported JSONL usage and OTEL usage rows, imported usage is treated as authoritative and overlapping OTEL usage rows are ignored for rollups. Raw events remain queryable in monitor/session history.
- Usage models are classified at query time into canonical model, provider, family, tier, lifecycle, and pricing-status fields. Unknown and deprecated models remain visible in responses.
- The Usage page’s Top Models **All tokens** view includes input, output, cache-read, and cache-write traffic. A persistent pricing warning names affected models and their observed token volume when pricing is unknown or known pricing has not yet been applied to historical zero-cost rows; it keeps cost totals explicitly non-estimated and directs the operator to `amon costs recalc` for the latter state.
- Usage endpoints accept optional `model`, `provider`, and `tier` filters in addition to date, project, and agent filters. Classification filters are applied consistently before summary, daily, attribution, tier, agent, and top-session panels aggregate.
- Usage summary includes `prior_total_cost_usd` and `cost_delta_pct` for the immediately preceding same-length date range when a valid current range is supplied.
- Usage budget reports live at `/api/v2/usage/budgets`. They read an optional local JSON config, reuse usage filters to compute current spend, and return alert states without blocking or enforcing agent activity.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@
return base;
}

function tokenTotal(row: UsageProjectBreakdown | UsageModelBreakdown | UsageTierBreakdown | UsageAgentBreakdown): number {
const base = row.input_tokens + row.output_tokens;
return kind === 'model' ? base + row.cache_read_tokens + row.cache_write_tokens : base;
}

function handleSelect(row: UsageProjectBreakdown | UsageModelBreakdown | UsageTierBreakdown | UsageAgentBreakdown): void {
if (kind === 'project' && 'project' in row) {
void usage.setProject(row.project === 'unknown' ? '' : row.project);
Expand Down Expand Up @@ -90,7 +95,7 @@
<div class="text-right">
<div class="tabular font-mono text-body text-ok">{formatCost(row.cost_usd)}</div>
<div class="mt-0.5 tabular font-mono text-meta text-text-faint">
{formatNumber(row.input_tokens + row.output_tokens)} tokens
{formatNumber(tokenTotal(row))} {kind === 'model' ? 'observed tokens' : 'tokens'}
</div>
</div>
</div>
Expand Down
6 changes: 4 additions & 2 deletions frontend/src/lib/components/usage/UsagePage.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import { usage } from '../../stores/usage.svelte';
import { Select } from '../ui';
import UsageCoverageBanner from './UsageCoverageBanner.svelte';
import UsageUnpricedWarning from './UsageUnpricedWarning.svelte';
import UsageSummaryCards from './UsageSummaryCards.svelte';
import UsageTimeline from './UsageTimeline.svelte';
import UsageBreakdownTable from './UsageBreakdownTable.svelte';
Expand All @@ -25,9 +26,9 @@
</script>

<main class="flex-1 overflow-y-auto p-4 sm:p-6 space-y-4">
<!-- Cost-specific facets; shared date/project/agent live in the Analytics bar. -->
<!-- Usage-specific facets; shared date/project/agent live in the Analytics bar. -->
<div class="flex flex-wrap items-center gap-2">
<span class="text-meta uppercase tracking-wide text-text-faint">Cost facets</span>
<span class="text-meta uppercase tracking-wide text-text-faint">Usage facets</span>
<Select
value={usage.provider}
options={usage.providerOptions}
Expand All @@ -51,6 +52,7 @@
/>
</div>

<UsageUnpricedWarning />
<UsageCoverageBanner />
<UsageSummaryCards />

Expand Down
4 changes: 2 additions & 2 deletions frontend/src/lib/components/usage/UsageTopModels.svelte
Original file line number Diff line number Diff line change
Expand Up @@ -92,12 +92,12 @@
<div>
<h3 class="text-h3">Top Models</h3>
<p class="mt-0.5 text-meta text-text-muted">
Model mix by day. Click a day to drill the page into it.
Model mix by day. All tokens include cache reads and writes. Click a day to drill the page into it.
</p>
</div>

<div class="flex shrink-0 rounded-sm border border-line" role="group" aria-label="Metric">
{#each [{ id: 'cost', label: 'Cost' }, { id: 'tokens', label: 'Tokens' }] as option}
{#each [{ id: 'cost', label: 'Cost' }, { id: 'tokens', label: 'All tokens' }] as option}
<button
class="px-2 py-1 text-meta transition-colors first:rounded-l-sm last:rounded-r-sm
{metric === option.id ? 'bg-surface-2 text-text' : 'text-text-faint hover:text-text-muted'}"
Expand Down
35 changes: 35 additions & 0 deletions frontend/src/lib/components/usage/UsageUnpricedWarning.svelte
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
<script lang="ts">
import { usage } from '../../stores/usage.svelte';
import { formatNumber } from '../../format';
import { summarizeIncompleteCostUsage } from './unpriced-usage';

const summary = $derived(summarizeIncompleteCostUsage(usage.models));
const warningMessage = $derived.by(() => {
if (!summary) return '';
if (summary.unknown_pricing_model_count > 0 && summary.unrecalculated_model_count > 0) {
return 'Some model pricing is unknown, and some historical costs have not been recalculated. Cost totals may be incomplete.';
}
if (summary.unknown_pricing_model_count > 0) {
return 'Model pricing is unknown. Cost totals may be incomplete until prices are added.';
}
return 'Known model pricing has not been applied to some historical usage. Cost totals may be incomplete until you run amon costs recalc.';
});
const modelPreview = $derived.by(() => {
if (!summary) return '';
const names = summary.models.slice(0, 3).join(', ');
const remaining = summary.models.length - 3;
return remaining > 0 ? `${names} +${remaining} more` : names;
});
</script>

{#if summary}
<div class="flex items-start gap-2 rounded-lg border border-warn/30 bg-warn/10 px-4 py-3 text-meta text-text-muted" role="status">
<span class="mt-1 inline-block h-1.5 w-1.5 shrink-0 rounded-full bg-warn" aria-hidden="true"></span>
<div>
<p class="text-text">{warningMessage}</p>
<p class="mt-1">
{formatNumber(summary.observed_tokens)} observed tokens across {formatNumber(summary.usage_events)} usage events from {formatNumber(summary.model_count)} model{summary.model_count === 1 ? '' : 's'}: <span class="font-mono text-text">{modelPreview}</span>
</p>
</div>
</div>
{/if}
8 changes: 6 additions & 2 deletions frontend/src/lib/components/usage/model-colors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ export interface ModelSliceLike {
cost_usd: number;
input_tokens: number;
output_tokens: number;
cache_read_tokens?: number;
cache_write_tokens?: number;
}

export interface ModelDailyPointLike {
Expand Down Expand Up @@ -47,7 +49,9 @@ if (TOP_N > SERIES_COLORS.length) {
export const METRICS: UsageMetric[] = ['cost', 'tokens'];

export function measure(slice: ModelSliceLike, metric: UsageMetric): number {
return metric === 'cost' ? slice.cost_usd : slice.input_tokens + slice.output_tokens;
return metric === 'cost'
? slice.cost_usd
: slice.input_tokens + slice.output_tokens + (slice.cache_read_tokens ?? 0) + (slice.cache_write_tokens ?? 0);
}

/** Models ranked over the whole range, so a model keeps its stack slot across days. */
Expand All @@ -70,7 +74,7 @@ export function rankModels(points: readonly ModelDailyPointLike[], metric: Usage
* A model's hue must survive the metric toggle, so colors cannot be handed out by
* rank — that repaints every survivor the moment the ranking changes. But they also
* cannot be keyed to the cost ranking alone: a token-heavy, cheap model can enter the
* Tokens top-N while sitting outside it, and would fall through to Other's gray,
* All-tokens top-N while sitting outside it, and would fall through to Other's gray,
* putting two different categories on one color.
*
* So color the union of every metric's top-N — every model that can actually appear —
Expand Down
43 changes: 43 additions & 0 deletions frontend/src/lib/components/usage/unpriced-usage.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
export interface ModelUsageLike {
model: string;
pricing_status: 'known' | 'deprecated' | 'unknown';
cost_usd: number;
input_tokens: number;
output_tokens: number;
cache_read_tokens: number;
cache_write_tokens: number;
usage_events: number;
}

export interface IncompleteCostUsageSummary {
model_count: number;
unknown_pricing_model_count: number;
unrecalculated_model_count: number;
usage_events: number;
observed_tokens: number;
models: string[];
}

export function observedTokens(model: Pick<ModelUsageLike, 'input_tokens' | 'output_tokens' | 'cache_read_tokens' | 'cache_write_tokens'>): number {
return model.input_tokens + model.output_tokens + model.cache_read_tokens + model.cache_write_tokens;
}

export function summarizeIncompleteCostUsage(models: readonly ModelUsageLike[]): IncompleteCostUsageSummary | null {
const affectedModels = models
.filter(model => (
model.pricing_status === 'unknown'
|| (model.cost_usd === 0 && observedTokens(model) > 0)
))
.sort((left, right) => observedTokens(right) - observedTokens(left) || left.model.localeCompare(right.model));

if (affectedModels.length === 0) return null;

return {
model_count: affectedModels.length,
unknown_pricing_model_count: affectedModels.filter(model => model.pricing_status === 'unknown').length,
unrecalculated_model_count: affectedModels.filter(model => model.pricing_status !== 'unknown').length,
usage_events: affectedModels.reduce((total, model) => total + model.usage_events, 0),
observed_tokens: affectedModels.reduce((total, model) => total + observedTokens(model), 0),
models: affectedModels.map(model => model.model),
};
}
10 changes: 9 additions & 1 deletion src/pricing/data/claude.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"provider": "anthropic",
"lastUpdated": "2026-07-08T00:00:00Z",
"lastUpdated": "2026-08-04T00:00:00Z",
"models": {
"claude-fable-5": {
"aliases": [],
Expand All @@ -10,6 +10,14 @@
"cacheWriteCostPerMTok": 12.5,
"deprecated": false
},
"claude-opus-5": {
"aliases": [],
"inputCostPerMTok": 5,
"outputCostPerMTok": 25,
"cacheReadCostPerMTok": 0.5,
"cacheWriteCostPerMTok": 6.25,
"deprecated": false
},
"claude-opus-4-8": {
"aliases": [],
"inputCostPerMTok": 5,
Expand Down
20 changes: 20 additions & 0 deletions tests/pricing.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,17 @@ describe('PricingRegistry', () => {
assert.equal(pricing.deprecated, false);
});

test('finds Claude Opus 5 and applies its published base and cache rates', () => {
const pricing = registry.lookup('claude-opus-5');
assert.ok(pricing);
assert.equal(pricing.provider, 'anthropic');
assert.equal(pricing.inputCostPerToken, 5 / 1_000_000);
assert.equal(pricing.outputCostPerToken, 25 / 1_000_000);
assert.equal(pricing.cacheReadCostPerToken, 0.5 / 1_000_000);
assert.equal(pricing.cacheWriteCostPerToken, 6.25 / 1_000_000);
assert.equal(pricing.deprecated, false);
});

test('finds Claude Fable 5 with the published per-MTok rates', () => {
const pricing = registry.lookup('claude-fable-5');
assert.ok(pricing);
Expand Down Expand Up @@ -224,6 +235,15 @@ describe('PricingRegistry', () => {
assert.equal(c.pricing_status, 'known');
});

test('classifies Claude Opus 5 as a known anthropic/opus tier', () => {
const c = classifyModel('claude-opus-5');
assert.equal(c.canonical_model, 'claude-opus-5');
assert.equal(c.provider, 'anthropic');
assert.equal(c.family, 'claude');
assert.equal(c.tier, 'opus');
assert.equal(c.pricing_status, 'known');
});

test('classifies the Antigravity flash id/display via gemini-3.5-flash', () => {
for (const id of ['gemini-3-flash-a', 'Gemini 3.5 Flash (Medium)']) {
const c = classifyModel(id);
Expand Down
95 changes: 95 additions & 0 deletions tests/unpriced-usage.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import assert from 'node:assert/strict';
import test from 'node:test';

import { summarizeIncompleteCostUsage } from '../frontend/src/lib/components/usage/unpriced-usage.js';

test('summarizes cache-inclusive usage for unknown-priced models', () => {
const summary = summarizeIncompleteCostUsage([
{
model: 'known-model',
pricing_status: 'known',
cost_usd: 0.01,
input_tokens: 100,
output_tokens: 50,
cache_read_tokens: 1_000,
cache_write_tokens: 10,
usage_events: 2,
},
{
model: 'small-unknown',
pricing_status: 'unknown',
cost_usd: 0,
input_tokens: 10,
output_tokens: 5,
cache_read_tokens: 20,
cache_write_tokens: 2,
usage_events: 3,
},
{
model: 'cache-heavy-unknown',
pricing_status: 'unknown',
cost_usd: 0,
input_tokens: 1,
output_tokens: 2,
cache_read_tokens: 10_000,
cache_write_tokens: 500,
usage_events: 7,
},
]);

assert.deepEqual(summary, {
model_count: 2,
unknown_pricing_model_count: 2,
unrecalculated_model_count: 0,
usage_events: 10,
observed_tokens: 10_540,
models: ['cache-heavy-unknown', 'small-unknown'],
});
});

test('does not warn when every model has known or deprecated pricing', () => {
assert.equal(summarizeIncompleteCostUsage([
{
model: 'known-model',
pricing_status: 'known',
cost_usd: 0.000005,
input_tokens: 1,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
usage_events: 1,
},
{
model: 'deprecated-model',
pricing_status: 'deprecated',
cost_usd: 0.000005,
input_tokens: 1,
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
usage_events: 1,
},
]), null);
});

test('retains known-priced usage with zero stored cost until recalculation', () => {
assert.deepEqual(summarizeIncompleteCostUsage([
{
model: 'claude-opus-5',
pricing_status: 'known',
cost_usd: 0,
input_tokens: 50,
output_tokens: 100,
cache_read_tokens: 1_000,
cache_write_tokens: 25,
usage_events: 4,
},
]), {
model_count: 1,
unknown_pricing_model_count: 0,
unrecalculated_model_count: 1,
usage_events: 4,
observed_tokens: 1_175,
models: ['claude-opus-5'],
});
});
16 changes: 16 additions & 0 deletions tests/usage-model-colors.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,22 @@ describe('usage top models: series selection', () => {
const points = [day(slice('opus', 5, 0), slice('ghost', 0, 0))];
assert.deepEqual(rankModels(points, 'cost'), ['opus']);
});

test('ranks cache-heavy unknown-priced models by all observed tokens', () => {
const points = [day(
slice('known-model', 10, 100),
{
model: 'unknown-model',
cost_usd: 0,
input_tokens: 1,
output_tokens: 1,
cache_read_tokens: 10_000,
cache_write_tokens: 500,
} as ModelDailyPointLike['models'][number],
)];

assert.equal(rankModels(points, 'tokens')[0], 'unknown-model');
});
});

describe('usage top models: color assignment', () => {
Expand Down