feat: telemetry — tracing spans with token/cost fields (C5)#65
Merged
Conversation
Phase C5 of the roadmap (final Phase C item): structured observability. - Spans: `agent_loop` (model) → `llm_stream` per turn (turn, model, and recorded-on-completion tokens_in/out/cached + cost_usd when the ModelConfig has pricing) → `tool` per execution (tool, tool_call_id, is_error). Durations come free with spans. - Futures instrumented with .instrument(span) — no entered guards held across .await. - OTel deliberately stays app-side (tracing-opentelemetry layer): the library gains no OTel dependency, spans compile to no-ops without a subscriber. Documented with an OTLP bridge snippet. - New examples/telemetry.rs (fmt subscriber with span timings; runs on MockProvider, no API key needed) — smoke-run verified. - tracing-subscriber added as a dev-dependency only. Tests: a capturing Layer + WithSubscriber-attached agent_loop run asserting the span set (1× agent_loop, 2× llm_stream for two turns, 1× tool). Docs: concepts/telemetry.md (+ SUMMARY), lib.rs/README bullets, CLAUDE.md, CHANGELOG. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
yuanhao
added a commit
that referenced
this pull request
Jul 10, 2026
Five-agent review of the combined v0.10.0...main diff (PRs #61-#65) surfaced convergent findings; this is the full fix batch. Critical fixes: - prompt_structured error handling overhauled: provider failures now surface as StructuredPromptError::Provider carrying the real error (previously laundered into Parse { raw: "" }); the raw-text scan is scoped to messages produced by THIS call (a failed run can no longer return an earlier turn's JSON as Ok(T)); the payload is taken from the LAST text block (tool-forcing appends after preamble); Parse now carries #[source] serde_json::Error. - Schema drop-leak closed structurally: the output_schema field on Agent is gone — the schema threads per-call through prompt_messages_internal into the run's AgentLoopConfig, so a timed-out/dropped future can't leave the agent schema-forced. - Bedrock replays Content::Thinking (with signature) as reasoningContent — previously captured then dropped, guaranteeing a ValidationException on multi-turn thinking + tool use. - Anthropic structured outputs disable thinking for that request (forced tool_choice + extended thinking is an API-level 400) with a warning. - Session::append_new verifies the history extends the current path and returns HistoryDiverged instead of silently dropping turns or corrupting the tree when compaction (on by default) rewrites agent messages; returns the appended count. - Session::from_jsonl single-pass validation: DuplicateId + UnknownParent (parents must precede children, which also makes cycles impossible — path_ids provably terminates). API shape (free now — these types are unreleased): - ToolMiddleware::before_tool takes a #[non_exhaustive] ToolCallRequest context struct instead of three frozen positional params. - StreamConfig and OutputSchema are #[non_exhaustive]; StreamConfig::new added (this release alone added output_schema to it). - ToolDecision documents the deliberate StopReason-style exhaustive policy. Hardening: - A panicking ToolMiddleware is contained (catch_unwind) as a Deny instead of killing the loop task and stripping the agent of its tools. - Middleware denials emit tracing::warn! (operator visibility). - unwrap_structured_tool_call removes only the synthetic tool call and preserves non-ToolUse stop reasons (Length truncation no longer laundered into success). - llm_stream span gains an error field; seek_checkpoint is latest-wins on duplicate labels; Vertex round-trips Gemini thought signatures on function calls (parity with google.rs). Docs: tools.md example on the new signature + panic note; structured-outputs error semantics + schema-dialect caveats + thinking caveat; telemetry OTel snippet updated to the current builder API (old one was removed in 0.17) and overhead claims made honest; session-trees compaction caveat + validation; "enforced natively where supported"; CHANGELOG Fixed section; CLAUDE.md. Tests (+15, 376 total): schema propagation via capturing provider (replaces the tautological reset test); Provider-error and stale-history-never-parsed; middleware never sees the synthetic tool; middleware panic containment; Batched-strategy deny; session divergence/dup-id/dangling-parent/latest-wins; bedrock thinking-replay body; anthropic structured-disables-thinking body; google thinking-drop pin; vertex signature round-trip; telemetry field values including exact cost math (on_record collector). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Phase C5 — the final Phase C item. The loop now emits structured
tracingspans, not just log lines:cargo run --example telemetryshows it live:Design
tracingspans and gains no OTel dependency — apps install thetracing-opentelemetrylayer and the same spans flow to Datadog/Tempo/Honeycomb/Jaeger. Zero overhead with no subscriber.cost_usdrecorded fromCostConfigwhen pricing is configured — cost attribution lands in dashboards for free..instrument(span); no entered guards across.await.tracing-subscriberadded as dev-dependency only (test + example).Tests (1 new, 361 total green)
Capturing
Layer+WithSubscriber-attachedagent_looprun asserting exact span counts (1× agent_loop, 2× llm_stream, 1× tool).Verification
clippy
-Dwarnings --all-featuresclean · 361 passed / 0 failed · docs-Dwarningsclean · example smoke-run verified · new mdBook pageconcepts/telemetry.mdwith the OTLP bridge snippet.🤖 Generated with Claude Code