feat(ingest): add Weights & Biases Weave adapter

[kaushikyelne]

Summary

Adds a new ingestion adapter for Weights & Biases Weave trace exports, extending the harness's observability platform coverage alongside the existing Braintrust, Phoenix, Langfuse, LangSmith, Mastra, and PostHog adapters.

Motivation

Weave is a widely adopted LLM observability and evaluation platform. Teams using Weave to trace their agent runs currently have no way to feed those traces into wmh for world-model building. This adapter closes that gap — a Weave user can now export their Calls (JSON/JSONL) and run wmh ingest run --source weave --file <export> to build a world model from their production data.

Design

Weave records agent executions as Calls (not OTLP spans). Each Call carries an op_name, inputs, output, trace_id, id, parent_id, and timestamps. Since this is a custom schema (not OpenTelemetry), the adapter follows the same pattern as posthog.py — it overrides spans_from_payload and emits SpanRecords using OTel GenAI attribute keys so the shared normalizer handles pairing and state extraction.

Key decisions

Decision	Rationale
Op-name heuristic classification	Weave has no span.kind tag. Calls are classified as LLM vs tool by checking if the op name contains markers like chat, complete, openai, anthropic, etc. Everything else is treated as a tool execution.
Weave URI parsing	Op names can be full URIs (weave:///entity/project/op/func_name:hash). The adapter strips the prefix and hash suffix to extract the human-readable function name.
Error detection via exception field	Weave stores errors in an exception string field (not a status code). The adapter checks for a non-empty exception or a status of "error"/"failed".
Live vendor pull	Implemented via Weave's /calls/stream_query endpoint (JSONL response). Requires WANDB_API_KEY and a --project entity/project argument.

Changes

File	What
wmh/ingest/weave.py	New WeaveAdapter — Call parsing, op-name classification, JSON/JSONL file loading, and live vendor pull via /calls/stream_query
wmh/ingest/weave_test.py	11 tests: op-name extraction, LLM classification heuristics, JSON round-trip, JSONL, error calls, plain completions, wrapper shapes, adapter registration, vendor pull validation
wmh/ingest/__init__.py	Register weave adapter on package import
examples/ingest/weave_to_wmh.sh	Usage example script for file-based and API-based ingestion

Testing

$ uv run pytest wmh/ingest/weave_test.py -v
11 passed in 0.67s

$ uv run ruff check .
All checks passed!

All tests are offline (file fixtures, no network). The adapter is SDK-free — it parses exported JSON directly and uses httpx for vendor pulls (same as PostHog, Langfuse).

Usage

# File-based (export from Weave UI → JSON/JSONL)
wmh ingest run --source weave --file weave_calls.json

# Live pull (requires WANDB_API_KEY)
export WANDB_API_KEY="your-key"
wmh ingest run --source weave --project "myteam/myproject" --limit 500

[greptile-apps]

Greptile Summary

This PR adds WeaveAdapter, a new ingestion adapter for Weights & Biases Weave Call exports, following the same pattern as the existing PostHogAdapter.

• Core adapter (wmh/ingest/weave.py): maps Weave Calls to SpanRecords via op-name heuristic classification (LLM vs. tool), handles URI parsing, JSON/JSONL/wrapper payloads, error detection via the exception field, and live pulls via /calls/stream_query. The parallel tool-call case (N > 1) correctly returns an empty span list for the LLM Call and lets each child Call produce a self-contained execute_tool Step, avoiding the action/observation mismatch fixed in the prior review round.
• Tests (wmh/ingest/weave_test.py): 11 offline tests covering op-name extraction, LLM heuristics, JSON and JSONL round-trips, error propagation, plain completions, the {"calls": [...]} wrapper shape, adapter registration, vendor-pull validation, and the new parallel tool-call scenario.
• Registration (wmh/ingest/__init__.py): adds weave to the import block and updates the docstring to list all seven provider adapters.

Confidence Score: 5/5

Safe to merge — the adapter correctly maps Weave Calls to SpanRecords, the parallel tool-call fix matches the posthog.py reference pattern, and all 11 tests are green and offline.

The core logic (op-name classification, JSON/JSONL loading, wrapper unwrapping, error detection, and live pull) is sound and closely mirrors the established posthog.py pattern. The parallel tool-call strategy is correct: returning an empty span list from the LLM Call and letting each child Call produce its own self-contained Step avoids the pairing mismatch documented in the prior review round. The only findings are a speculative 'data' wrapper key with no documented Weave source and no test, and the corresponding missing 'results' wrapper test — neither affects runtime behavior.

wmh/ingest/weave.py and wmh/ingest/weave_test.py for the minor wrapper-shape coverage gap noted above.

Important Files Changed

Filename	Overview
wmh/ingest/weave.py	New WeaveAdapter: op-name classification, JSON/JSONL loading, and live vendor pull; parallel tool-call fix correctly emits zero LLM-side spans so each child Call produces a self-contained Step.
wmh/ingest/weave_test.py	11 tests covering op-name extraction, LLM heuristics, JSON/JSONL round-trips, error calls, plain completions, wrapper shapes, adapter registration, vendor-pull validation, and parallel tool calls; all offline.
wmh/ingest/init.py	Registers weave adapter on package import; docstring updated to list all seven provider adapters alphabetically.
examples/ingest/weave_to_wmh.sh	Usage example script for file-based ingestion; straightforward and safe.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant F as File or Weave API
    participant WA as WeaveAdapter
    participant WS as _weave_spans
    participant N as normalize

    F->>WA: from_file / from_vendor
    WA->>WA: _load_payloads (JSON or JSONL)
    loop each payload
        WA->>WA: "spans_from_payload -> _extract_calls"
        loop each Weave Call
            WA->>WS: _weave_spans(call, ordinal)
            alt tool call
                WS-->>WA: execute_tool SpanRecord
            else LLM no tool calls
                WS-->>WA: chat SpanRecord with completion
            else LLM single tool call
                WS-->>WA: chat SpanRecord with action
            else LLM parallel tool calls
                WS-->>WA: empty list, child Calls emit own tool spans
            end
        end
    end
    WA->>N: spans_to_traces(all_spans)
    N->>N: group by trace_id, sort by start_nano
    N->>N: pair LLM spans to following tool spans
    N-->>WA: list of Trace
    WA-->>F: list of Trace

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant F as File or Weave API
    participant WA as WeaveAdapter
    participant WS as _weave_spans
    participant N as normalize

    F->>WA: from_file / from_vendor
    WA->>WA: _load_payloads (JSON or JSONL)
    loop each payload
        WA->>WA: "spans_from_payload -> _extract_calls"
        loop each Weave Call
            WA->>WS: _weave_spans(call, ordinal)
            alt tool call
                WS-->>WA: execute_tool SpanRecord
            else LLM no tool calls
                WS-->>WA: chat SpanRecord with completion
            else LLM single tool call
                WS-->>WA: chat SpanRecord with action
            else LLM parallel tool calls
                WS-->>WA: empty list, child Calls emit own tool spans
            end
        end
    end
    WA->>N: spans_to_traces(all_spans)
    N->>N: group by trace_id, sort by start_nano
    N->>N: pair LLM spans to following tool spans
    N-->>WA: list of Trace
    WA-->>F: list of Trace

Loading

_{Reviews (2): Last reviewed commit: "feat(ingest): add Weights & Biases Weave..." | Re-trigger Greptile}

[kondent-entreprise]

Excellent addition, Weave coverage closes a real gap. Most production agent teams I know are on Weave for tracing, and until now they had to re-export to OTLP or hand-roll a converter. This keeps the ingest surface consistent with the other six adapters.

What lands well

area | why it works -- | -- Op-name heuristic | Weave has no span.kind, so classifying by chat, complete, openai, anthropic in the op name mirrors what we do in posthog.py and avoids a hard dependency on Weave SDK URI parsing | stripping weave:///entity/project/op/func:hash to human name keeps traces readable in wmh UI Error detection | checking exception string plus status in ["error","failed"] matches Weave's actual export schema, not OTel status codes Parallel tool calls | returning empty span list for the LLM Call and letting each child Call emit its own execute_tool Step avoids the action/observation mismatch we hit in PR #116 review. This is the correct pattern

Tests look solid: 11 offline tests covering op-name extraction, LLM heuristics, JSON and JSONL round-trips, error propagation, plain completions, {"calls": [...]} wrapper, registration, vendor pull validation, and parallel tool calls. All green in 0.67s.

Two small nits before merge

1. Wrapper coverage gap — you handle {"calls": [...]} but Greptile notes a speculative {"data": [...]} and missing {"results": [...]} wrapper. Weave UI export sometimes wraps in results when you export a filtered view. Add one test:

Python

def test_results_wrapper():    payload = {"results": [sample_call()]}    spans = WeaveAdapter().spans_from_payload(payload)    assert len(spans) == 1

2. Live pull pagination — /calls/stream_query returns JSONL but respects limit server-side only for first page. If a user passes --limit 5000, you will stream indefinitely. Consider:

Python

# in from_vendorparams = {"limit": min(limit, 1000), "$offset": 0}while len(collected) < limit:    ...

Nice-to-have for follow-up

• Weave stores attributes and feedback on Calls. Mapping attributes.model → gen_ai.request.model and feedback → span events would give us eval signals for free in the world model.
• Add WANDB_API_KEY validation early with a clear error: SystemExit("WANDB_API_KEY not set, export it or use --file") — same pattern as Langfuse adapter.

Usage is clean:

Bash

# filewmh ingest run --source weave --file weave_calls.json
# liveexport WANDB_API_KEY=...wmh ingest run --source weave --project "myteam/myproject" --limit 500

1 line hidden

Safe to merge as-is, confidence 5/5 from Greptile matches my read. This plus PostHog and Langfuse means we now cover ∼90% of the observability stacks I see in the wild.