feat(agent-visualizer): show agent model names - #60
Conversation
|
|
|
Hi @patoles, thanks for this repository, it has been very useful for understanding how agents move through a flow. I added a small improvement to show which model is running for each subagent. I think this can make it easier to understand and debug agent executions. Curious to hear what you think. Any feedback is welcome! |
model_detected was guarded by !session.model, so only the orchestrator ever received its model. Subagents never got theirs because the session- level flag was already set. Track detection per agent with a Set instead.
…g sessions Sessions created before the new field was added crash on .has() because modelDetectedAgents is undefined. Lazily initialize the Set on first access so hot-reloaded relays and older session objects work correctly.
…ith state The model label was wrapping to two lines when placed inline after the state text. Move it under the agent name (same pattern as the detail card) and push the state label to the right with ml-auto.
patoles
left a comment
There was a problem hiding this comment.
Thank you very much for this! It's a feature I'm really glad to see land, and you found exactly the right seam: the frontend was already prepared for it (Agent.model exists and handleModelDetected is keyed by agent), so per-agent emission is the correct missing piece. The detail card wiring works, the tests are appreciated, and they're picked up by the root pnpm test glob. I did find a few things worth addressing before merge. It looks like a long list, but the first three all share one root cause, so it's smaller than it appears.
1. The once-ever Set dedup loses/freezes models (root cause of three issues)
modelDetectedAgents: Set<string> records that an agent's model was emitted, but not which model, and it's never cleared or replayed. Three consequences:
- Panel reopen permanently loses subagent models.
replaySessionStartre-emits the orchestrator spawn withsession.modeland resetssub.spawnEmittedso subagent spawns replay, but subagent spawn payloads carry no model, and the Set keeps suppressingmodel_detectedre-emission. Close/reopen the visualizer mid-session and every subagent's model row is blank (withtokensMaxfalling back) for the rest of the session. - Mid-session model switches never show. After an agent's first emission,
/modelswitches (or a re-dispatched same-named subagent with a differentmodel:override) are suppressed forever, leaving a stale label and a wrong context-window %. - The one-shot can be consumed before the frontend ever sees it. On re-attach, prescan adds subagent names to
spawnedSubagentswithout emittingagent_spawn(transcript-parser.ts:449-452). Ifmodel_detectedthen fires before the frontend knows the agent,handleModelDetecteddrops it silently, and the Set entry means it never retries.
The codebase already has the right pattern: the Codex parser tracks lastEmittedModel and re-emits on change (codex-rollout-parser.ts:80, with test coverage). Suggest replacing the Set with Map<agentName, model> and emitting when the value differs. Same footprint, subsumes the once-per-agent behavior, keeps the two watchers consistent, and gives replaySessionStart the data to re-deliver per-agent models on reconnect (mirroring what it already does for session.model).
2. scripts/relay.ts constructs WatchedSession too and wasn't updated
The new field is required in protocol.ts, but relay.ts:195 builds its own WatchedSession literal (it already initializes spawnedSubagents: new Set() right there). Nothing typechecks scripts/ (the extension's tsc --noEmit only covers extension/src), so this ships as a silent interface violation, and the lazy-init in the parser (session?.modelDetectedAgents ?? (session ? (session.modelDetectedAgents = new Set()) : undefined)) is what keeps the standalone dashboard from crashing at runtime. Easy to miss; I only caught it because I knew the relay had its own copy. Adding modelDetectedAgents: new Set() to the relay's literal lets the parser gate collapse to a plain !session.modelDetectedAgents.has(agentName), which also stops the field from looking optional when it isn't.
3. formatModelName: Bedrock IDs render the date as a minor version
The date strip /-\d{8}$/ is end-anchored, so any suffix after the date defeats it, and CLAUDE_NEW's optional (?:-(\d+))? swallows the date:
formatModelName('us.anthropic.claude-sonnet-4-20250514-v1:0') // → "Sonnet 4.20250514"
(IDs with a minor version, like ...opus-4-1-20250805-v1:0 → "Opus 4.1", happen to work, which hides the broken case.) Making the date strip non-anchored (e.g. /-\d{8}(?=$|[-@:.])/) or stripping known provider prefixes/suffixes first would cover Bedrock/Vertex IDs. Worth a test case too.
Smaller points in the same function:
- The GPT regex is unanchored and matched against raw
modelinstead ofbase, sochatgpt-4o-latestbecomes "GPT-4o-latest". Anchor it (word boundary or^) and match againstbaselike the Claude branches. - The two Claude branches are copy-paste with capture indices swapped. A tiny shared helper (
claudeLabel(family, major, minor?)) removes the drift risk. web/lib/canvas-constants.tsalready maintains Claude family patterns for context size and cost, both matched against lower-cased IDs. This adds a third lockstep-edit family list, matched case-sensitively. Consider deriving all three from one shared family constant and lowercasing before matching.
4. PR description vs. diff
The description says the model shows "in the agent detail card and chat panel" and that the model is "passed through the visualizer panel props", but chat-panel.tsx isn't touched (its props have no model) and no props were added anywhere; the detail card works because selectedAgent already carries model. Was a commit missed, or should the description be trimmed to the detail card?
The Map-on-change and the relay init are the two load-bearing ones; the formatModelName items are quick tweaks on top. Happy to talk through any of it if you see it differently, and thanks again for the contribution!
…nge detection Replace modelDetectedAgents Set<string> with Map<string, string> so the extension tracks which model was last emitted per agent, not just whether any model was emitted. This fixes three issues: panel reopen losing subagent models, mid-session /model switches being silently dropped, and the one-shot emission being consumed before the frontend connects. replaySessionStart now re-emits model_detected for every entry in the Map, matching the change-detection pattern the Codex parser already uses. Also adds the missing modelDetectedAgents field to the relay's WatchedSession literal (scripts/relay.ts).
Strip provider prefixes (us.anthropic. etc.) and trailing version suffixes (-v1:0) before date removal so Bedrock IDs like us.anthropic.claude-sonnet-4-20250514-v1:0 correctly resolve to "Sonnet 4" instead of "Sonnet 4.20250514". Also: use non-anchored date stamp pattern, match GPT regex against the cleaned base string, and extract a shared claudeLabel helper to remove copy-paste between the two Claude branches. Adds test coverage for Bedrock prefixed IDs and additional edge cases.
…ents replaySessionStart emitted model_detected for all agents before subagents existed in the frontend, so handleModelDetected silently dropped the events. Re-emit subagent spawns inline (with model in payload) before the model_detected loop so the frontend has all agents registered when model events arrive.
patoles
left a comment
There was a problem hiding this comment.
Nice work on the revision, this addressed everything I was worried about, and the replay handling is better than what I sketched. I took the liberty of updating the PR description to match the final diff (the chat panel part didn't make it into the change set, which is fine, the detail card is the useful surface anyway). The shared family-list idea I mentioned is small enough that I'll do it myself in a follow-up. Thanks for the thorough turnaround!
The family list existed in three places: the formatModelName regexes, MODEL_FAMILY_CONTEXT, and MODEL_FAMILY_COST. Adding a family meant three lockstep edits across two files, and the formatter matched case-sensitively while the tables are documented as lower-cased. One CLAUDE_FAMILIES table now drives display names, context windows, and cost rates; the formatter regexes are built from it and match case-insensitively. Follow-up promised in #60.
What does this PR do?
Mapkeyed by agent name), re-emittingmodel_detectedwhen the model changes (e.g./modelswitch), and replays per-agent models when the webview reconnects.How to test
pnpm run devand open the visualizer.claude-3-5-sonnet-20241022andus.anthropic.claude-opus-4-1-20250805-v1:0render asSonnet 3.5andOpus 4.1.pnpm testandpnpm run build:web.Preview
Checklist