Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,31 @@ adheres to [Semantic Versioning](https://semver.org/).

## Unreleased

### Fixed (pre-release review of the items below)

- `prompt_structured` now surfaces provider failures as
`StructuredPromptError::Provider` (previously laundered into
`Parse { raw: "" }`), scans only messages produced by the current call
(never stale history), and threads the schema per-call so a dropped/timed-
out future can't leave the agent stuck in schema-forcing mode.
- Bedrock replays `Content::Thinking` blocks (with signatures) on subsequent
requests — previously captured and then dropped, breaking multi-turn
thinking + tool use with a ValidationException.
- Anthropic structured outputs disable extended thinking for that request
(forced tool choice + thinking is an API-level conflict) with a warning.
- Vertex now round-trips Gemini thought signatures on function calls (parity
with the Gemini API provider).
- `Session::append_new` verifies the history still extends the session path
and returns `HistoryDiverged` instead of silently corrupting the tree (the
usual cause: context compaction); `from_jsonl` validates ids and parents
(duplicates/dangling/cycles rejected); `seek_checkpoint` is latest-wins.
- A panicking `ToolMiddleware` is contained as a denial instead of killing
the loop task (which stripped the agent of its tools).
- Middleware denials are logged (`tracing::warn!`) so operators see them.
- `ToolMiddleware::before_tool` takes a `ToolCallRequest` context struct
(extensible without breaking implementors); `StreamConfig` and
`OutputSchema` are `#[non_exhaustive]` with constructors.

### Added

- **Telemetry** — `tracing` spans: `agent_loop` (model), `llm_stream` per
Expand All @@ -23,7 +48,7 @@ adheres to [Semantic Versioning](https://semver.org/).

- **Session trees** — `Session`: branching conversation history with
`append`/`seek`/`checkpoint`, fork-preserving edits, `path_messages()` for
branch resume, and JSONL persistence. The pi-style id/parentId tree; maps
branch resume, and JSONL persistence. The pi-style id/parent_id tree; maps
to GASP's `transcripts/` tier.

- **Structured outputs** — `Agent::prompt_structured::<T>(text, schema)`
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,9 @@ The loop: stream assistant response → extract tool calls → execute tools (pa
- `Sequential` — one at a time, checks steering queue between each
- `Batched { size }` — concurrent within batch, steering check between batches

**Structured outputs** (`Agent::prompt_structured::<T>(text, schema)`): the schema travels via `StreamConfig.output_schema` (`OutputSchema` in `provider/traits.rs`). Anthropic enforces it by tool-forcing (a synthetic tool + `tool_choice`; the loop's `unwrap_structured_tool_call` converts the forced call back to text **before** tool-call extraction); OpenAI-compat uses `response_format: json_schema`; Gemini uses `responseSchema`. Responses/Azure/Vertex/Bedrock warn and ignore.
**Structured outputs** (`Agent::prompt_structured::<T>(text, schema)`): the schema is threaded per-call through `prompt_messages_internal` into `StreamConfig.output_schema` (`OutputSchema` in `provider/traits.rs`) — never stored on the Agent. Errors: `Provider` (API failure), `Parse { source, raw }`, `NoOutput`; only this call's messages are scanned. Anthropic enforces it by tool-forcing (a synthetic tool + `tool_choice`; the loop's `unwrap_structured_tool_call` converts the forced call back to text **before** tool-call extraction); OpenAI-compat uses `response_format: json_schema`; Gemini uses `responseSchema`. Responses/Azure/Vertex/Bedrock warn and ignore.

**Tool middleware** (`ToolMiddleware` in `types.rs`): async approve/deny/modify hooks gating every tool call, run in a chain at the single choke point (`execute_single_tool`) shared by all three strategies. `Deny(reason)` becomes an error tool result the LLM sees (loop continues); `Modify(args)` rewrites the call. Installed via `Agent::with_tool_middleware` / `SubAgentTool::with_tool_middleware` / `AgentLoopConfig::tool_middleware`. Empty chain = allow all.
**Tool middleware** (`ToolMiddleware` in `types.rs`): async approve/deny/modify hooks gating every tool call, run in a chain at the single choke point (`execute_single_tool`) shared by all three strategies. `before_tool` takes a `#[non_exhaustive]` `ToolCallRequest` context struct (extensible without breaking implementors). `Deny(reason)` becomes an error tool result the LLM sees (loop continues); `Modify(args)` rewrites the call; a panicking middleware is contained as a denial. Installed via `Agent::with_tool_middleware` / `SubAgentTool::with_tool_middleware` / `AgentLoopConfig::tool_middleware`. Empty chain = allow all.

### OpenAPI Integration (`openapi/`, feature-gated)

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Everything is observable via events. Supports 7 API protocols covering 20+ LLM p
- Parallel tool execution by default — sequential and batched strategies also available
- Sub-agents via `SubAgentTool` — delegate tasks to child agent loops with their own tools and system prompts
- Tool middleware — async approve/deny/modify hooks gating every tool call (`with_tool_middleware`), the mechanism for permission prompts and policy engines
- Structured outputs — `prompt_structured::<T>()` returns typed, schema-validated replies, enforced natively per provider (Anthropic tool-forcing, OpenAI `json_schema`, Gemini `responseSchema`)
- Structured outputs — `prompt_structured::<T>()` returns typed, schema-validated replies, enforced natively where supported (Anthropic tool-forcing, OpenAI `json_schema`, Gemini `responseSchema`; other protocols log a warning)
- Real-time event streaming — `prompt()` spawns the loop concurrently and returns events immediately; `prompt_with_sender()` accepts a caller-provided channel for custom consumption
- Streaming tool output — tools emit real-time progress via `on_update` callback
- Multimodal support — `Content::Image` flows through tool results across all providers
Expand All @@ -45,7 +45,7 @@ Everything is observable via events. Supports 7 API protocols covering 20+ LLM p
- State persistence — `save_messages()` / `restore_messages()` for pause/resume workflows
- Session trees — branching history with fork, checkpoints, and JSONL persistence (`Session`); edit an earlier turn and re-run without losing the original branch
- Lifecycle callbacks — `before_turn`, `after_turn`, `on_error` for observability and control
- Telemetry — `tracing` spans for the loop, each LLM stream (tokens + cost fields), and each tool execution; bridge to OpenTelemetry via `tracing-opentelemetry`, zero overhead when no subscriber is installed
- Telemetry — `tracing` spans for the loop, each LLM stream (tokens + cost fields), and each tool execution; bridge to OpenTelemetry via `tracing-opentelemetry`, negligible overhead when no subscriber is installed
- Full serde support — all core types implement `Serialize`/`Deserialize`/`PartialEq`
- [AgentSkills](https://agentskills.io)-compatible skills — load skill directories, inject into system prompt, agent activates on demand

Expand Down
16 changes: 12 additions & 4 deletions docs/concepts/session-trees.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "So
let mut rx = agent.prompt("draft a plan").await;
while rx.recv().await.is_some() {}
agent.finish().await;
session.append_new(agent.messages());
session.append_new(agent.messages())?;
session.checkpoint("first-draft")?;

// ... more turns ...
Expand All @@ -32,7 +32,7 @@ let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "So
let mut rx = agent.prompt("actually, make it a library instead").await;
while rx.recv().await.is_some() {}
agent.finish().await;
session.append_new(agent.messages()); // new branch recorded
session.append_new(agent.messages())?; // new branch recorded

// Both branches exist:
assert_eq!(session.branch_tips().len(), 2);
Expand All @@ -46,15 +46,23 @@ let restored = Session::from_jsonl(&std::fs::read_to_string("session.jsonl")?)?;
```

One entry per line, append-friendly, diff-friendly. On load the head is the
last line's entry. The flat `save_messages()` / `restore_messages()` API
last line's entry, and the file is validated (duplicate ids and
dangling/forward parent references are rejected — which also makes cycles
impossible).

> **Compaction caveat:** `append_new` verifies the agent's history still
> extends the session path and returns `SessionError::HistoryDiverged` when it
> doesn't. The usual cause is context compaction (on by default) rewriting the
> agent's messages — disable context management on session-tracked agents, or
> rebuild the branch with `Session::from_messages` after a divergence. The flat `save_messages()` / `restore_messages()` API
remains for single-branch persistence.

## API sketch

| Method | Purpose |
|---|---|
| `append(msg) -> id` | Add as child of head, advance head |
| `append_new(&agent_messages)` | Record everything beyond the current path the post-run sync |
| `append_new(&agent_messages)` | Record everything beyond the current path (verifies the prefix; errors on divergence) |
| `seek(id)` / `seek_checkpoint(label)` | Move the head (fork point) |
| `checkpoint(label)` | Label the head |
| `path_messages()` | Root→head messages — feed to `Agent::with_messages` |
Expand Down
19 changes: 14 additions & 5 deletions docs/concepts/structured-outputs.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,11 @@ let invoice: Invoice = agent
```

Derive the schema however you like — by hand as above, or with the
[`schemars`](https://crates.io/crates/schemars) crate
(`schemars::schema_for!(Invoice)`).
[`schemars`](https://crates.io/crates/schemars) crate (convert with
`serde_json::to_value(schemars::schema_for!(Invoice))`). Mind the provider
dialects: OpenAI strict mode requires `additionalProperties: false` and every
property listed in `required`; Gemini rejects `$defs`/`$ref`. Schemas are
passed through as given.

## How each provider enforces it

Expand All @@ -48,9 +51,15 @@ Derive the schema however you like — by hand as above, or with the

- `prompt_structured` runs the loop to completion internally and returns the
parsed `T` — there is no event receiver for this call.
- On parse failure you get `StructuredPromptError::Parse { error, raw }` — the
raw model text is preserved so you can retry or salvage.
- On Anthropic the forced tool call preempts regular tools for that request.
- Three error shapes: `Provider { message }` when the API call itself failed
(auth, network, a schema-induced 400 — retrying the parse is pointless);
`Parse { source, raw }` when the model's text didn't deserialize (the raw
text is preserved so you can retry or salvage); `NoOutput` when the run
produced no text. Only messages produced by **this call** are considered —
stale output from earlier turns is never parsed.
- On Anthropic the forced tool call preempts regular tools for that request,
and **disables extended thinking** for that request (forced tool choice and
thinking are mutually exclusive at the API level — a warning is logged).
Treat structured prompts as **extraction/finalization calls**, not agentic
tool-using turns.
- Markdown code fences around the JSON are stripped defensively before
Expand Down
26 changes: 17 additions & 9 deletions docs/concepts/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

yoagent instruments the loop with [`tracing`](https://docs.rs/tracing) **spans**
— structured, timed, nested units your observability stack can consume. With
no subscriber installed they compile to no-ops; there is zero overhead unless
you opt in.
no subscriber installed the overhead is near-zero (a cached per-callsite
interest check); nothing is exported unless you opt in.

## The span tree

```
agent_loop (model)
└─ llm_stream (turn, model, tokens_in, tokens_out, tokens_cached, cost_usd)
└─ llm_stream (turn, model, tokens_in, tokens_out, tokens_cached, cost_usd, error)
└─ tool (tool, tool_call_id, is_error)
```

Expand Down Expand Up @@ -38,19 +38,27 @@ same spans flow to any OTLP backend — Datadog, Grafana Tempo, Honeycomb,
Jaeger:

```rust
use opentelemetry_otlp::WithExportConfig;
// opentelemetry_otlp 0.27+, opentelemetry_sdk 0.27+, tracing-opentelemetry 0.28+
use opentelemetry::trace::TracerProvider as _;
use tracing_subscriber::layer::SubscriberExt;

let tracer = opentelemetry_otlp::new_pipeline()
.tracing()
.with_exporter(opentelemetry_otlp::new_exporter().tonic())
.install_batch(opentelemetry_sdk::runtime::Tokio)?;
let exporter = opentelemetry_otlp::SpanExporter::builder()
.with_tonic()
.build()?;
let provider = opentelemetry_sdk::trace::SdkTracerProvider::builder()
.with_batch_exporter(exporter)
.build();

let subscriber = tracing_subscriber::registry()
.with(tracing_opentelemetry::layer().with_tracer(tracer));
.with(tracing_opentelemetry::layer().with_tracer(provider.tracer("yoagent-app")));
tracing::subscriber::set_global_default(subscriber)?;
```

(The OTel crates rework their builder APIs between releases — if this snippet
drifts, the authoritative wiring is the
[`tracing-opentelemetry` docs](https://docs.rs/tracing-opentelemetry); yoagent
only emits standard `tracing` spans and does not depend on OTel.)

Because these are ordinary `tracing` spans, an agent call nests inside your
app's existing request traces (e.g. an axum handler span) automatically.

Expand Down
15 changes: 7 additions & 8 deletions docs/concepts/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -460,19 +460,14 @@ behind permission prompts, policy engines, and argument rewriting. yoagent
ships the hook, not a policy: with no middleware installed, every call runs.

```rust
use yoagent::{ToolDecision, ToolMiddleware};
use yoagent::{ToolCallRequest, ToolDecision, ToolMiddleware};

struct ReadOnlyPolicy;

#[async_trait::async_trait]
impl ToolMiddleware for ReadOnlyPolicy {
async fn before_tool(
&self,
_tool_call_id: &str,
tool_name: &str,
_args: &serde_json::Value,
) -> ToolDecision {
match tool_name {
async fn before_tool(&self, call: &ToolCallRequest<'_>) -> ToolDecision {
match call.tool_name {
"write_file" | "edit_file" | "bash" => {
ToolDecision::Deny("read-only session".into())
}
Expand All @@ -498,6 +493,10 @@ Semantics:
adapt — pick another tool, ask the user — and the loop continues. A denial
never aborts the run.

A middleware that panics is contained: the call is denied (reason `"tool
middleware panicked"`) and the loop continues — a buggy policy can't kill the
run.

The hook is `async`, so an interactive app can prompt a human before deciding.
Note that under the default `Parallel` execution strategy, middleware for
parallel tool calls run concurrently — if you need one-at-a-time approval UX,
Expand Down
Loading
Loading