feat: wire lex-log into agent/tool execution for distributed tracing

[alpibrupa]

Current state

lex-trail is integrated (session events logged via trail_log). lex-log is NOT integrated — agent execution, tool calls, and LLM steps produce no spans or metrics.

Gap

lex-code is observability-dark at runtime:

• No span per agent turn (can't measure latency, identify slow tools)
• No metrics on tool success/failure rates
• No distributed trace correlation for multi-agent scenarios (multi_agent.lex)
• No log correlation between LLM steps and trail events

This means humans monitoring deployed lex-code instances can't verify runtime behaviour without reading logs — undermining the trust model.

What to add

Using lex-log/instrument.lex (already exists):

1. Per-turn span in session.lex

let scope := instrument.a2a_begin(ctx, session.id)
let result := run_turn_impl(session, input, provider_tag)
let scope2 := instrument.a2a_finish(scope, result.ok)

2. Per-tool-call span in agent dispatch

Each tool execution in lex-llm/agent.lex's dispatch loop should emit:

• span.start("tool.execute") with tool_name attr
• span.finish on completion with success attr

3. Metrics

• counter("tool.calls", [("tool", name), ("success", "true|false")])
• histogram("turn.duration_ms") per agent turn

4. Multi-agent trace propagation

multi_agent.lex spawns impl + test agents concurrently. Each should carry the parent trace context so their spans appear as children in the distributed trace.

Configuration

OTel export should be opt-in via env: LEX_OTLP_ENDPOINT=http://localhost:4318 (stdout JSON-lines if unset).

[alpibrupa]

Reviewed this against the current tree before starting. The premise holds; three of the four items need reframing.

Still true

grep -rn 'lex-log' src/ returns nothing, and lex-log isn't in lex.toml (it's only in the package cache, pulled transitively). No spans, no metrics, no OTLP. A human watching a deployed instance still can't see turn latency or tool success rates.

1. The code sketch doesn't compile

let scope := instrument.a2a_begin(ctx, session.id)

The real signature takes four arguments, the first a Str, not a context:

fn a2a_begin(traceparent :: Str, task_id :: Str, capability :: Str, service :: Str) -> [random, time] Scope

run_turn_impl also doesn't exist. The turn functions are run_turn_with_provider / run_turn_streaming_with_provider, with finish_turn as the shared tail — that's the natural seam, not a wrapper around an impl function.

2. Per-tool-call spans are largely already built — in a different representation

dispatch_one_traced (lex-llm src/agent.lex:478) already emits, for every tool call:

• cap_invoked with tool name and args — the span start
• cap_completed / cap_failed — the end, carrying success
• both parented to the invoke event via inv_id — causal parenting
• spec_denied when the permission spec rejects the call

And Event = { id, kind, parent, payload_json, ts_ms }, readable via log.range(from_ms, to_ms). That is a timestamped, parent-linked span tree already. What it lacks is the OTel wire format, not the instrumentation.

Adding a second span.start/span.finish stream inside the same loop would mean two systems recording the same facts and drifting apart — which is exactly how #32 failed: the loop wrote to one place, the reader read another, and the tool reported the gap as "no attestations".

So: project the trail into spans and export that, rather than instrument twice. The trail supplies id, parent, name, attributes, and both timestamps; exporter.export_spans(cfg, spans) takes a plain List[span.Span].

There's a bonus in doing it this way. TraceCtx = { trace_id, span_id, sampled } is a plain record, and trail event ids are sha256_str hashes. Deriving span ids from event ids means no random effect anywhere in the export path, and span ids that are stable across re-exports — re-exporting a session produces the same trace, rather than a new one each time. That matches how the rest of the system is content-addressed.

3. Metrics — unchanged

meter.counter / meter.histogram exist and match the issue. Both derive from the same trail projection: tool.calls from cap_completed vs cap_failed, turn.duration_ms from the turn's own span.

4. Multi-agent propagation — real, but not via traceparent threading

multi_agent.lex still runs run_parallel under [concurrent], so the need is genuine. But each sub-agent opens its own session and its own trail log, so there is no call path to thread a traceparent along. Correlation has to happen where the logs are joined, by deriving each child's trace_id from the parent task rather than passing a header down.

Effect rows

Favourable, unlike #32. run_turn_with_provider is [env, net, llm, io, proc, sql, time, approval] and finish_turn is [io, sql, time]; export needs net and env added to finish_turn, both already in the caller's row. run_turn is called from 8 sites and stored in no record field, so §1.6 invariance doesn't bite, and serve_web's net.serve_fn handler already declares everything needed.

Proceeding on the reframed shape: a trail→OTel projection exported at turn end, opt-in via LEX_OTLP_ENDPOINT, stdout JSON-lines otherwise.

---

Generated by Claude Code