bullpen run --json: consume a run as a stream instead of scraping a terminal

[Steel-tech]

Closes #9.

What was impossible

Nothing could drive bullpen run programmatically. The final answer reached
stdout as unframed text deltas, tool activity was human prose behind -v on
stderr, and the Event channel the loop already publishes had three of its
four variants dropped on the floor by the CLI printer. Worst of all, a
consumer could only learn a run had finished by watching stdout close — which
looks exactly like a crash, a killed process, or a broken pipe.

With --json, stdout is newline-delimited JSON and nothing else: one object
per event, written and flushed as the event happens, with a terminal result
object carrying the session id, the final text, cumulative usage, an error
boolean and the provider message. Success and failure both produce it, and it
is always last. Prose, the [session … tokens] footer, the recovery notice
and -v tool activity all stay on stderr, so --json and -v compose.

Without the flag, output is unchanged.

Decisions a reviewer would otherwise have to reverse-engineer

The kind strings are hand-written, not derived. KIND_ASSISTANT_TEXT,
KIND_TOOL_START, KIND_TOOL_END, KIND_TURN_DONE, KIND_RESULT,
KIND_DISPATCHED are const &str in crates/cli/src/json.rs, and the tests
assert them as string literals. No stringify!, no serde variant-name
derivation, no #[derive(Serialize)] on Event. bullpen_agent::Event is
internal state; the wire is a compatibility surface the moment anything reads
it, so the CLI owns the names and an internal rename cannot move them.
event_json matches Event exhaustively, so a fifth variant is a compile
error rather than a silent gap in the stream.

The text-delta sink stays attached and its text is discarded. Detaching
it would flip the agent from complete_streaming to complete — a real
behavior change smuggled in behind an output flag. Text deltas are not
emitted as JSON; the per-turn AssistantText event is the only text event, so
the stream stays one-object-per-semantic-event rather than a token firehose.

Tool-payload capping is required, not incidental. run_tool_with_events
emits ToolEnd with the uncapped output; cap_result only runs as a result
enters the transcript. Rather than duplicate the literal, this makes
MAX_TOOL_RESULT_BYTES pub so both paths share one number.
tool_end.output truncates on a char boundary with an output_truncated
flag; tool_start.input is the JSON value verbatim when its serialization
fits and the truncated serialization string plus input_truncated when it
does not. The result object's text is deliberately not capped — the
issue caps tool inputs and outputs, and truncating the final answer would
silently corrupt the deliverable.

error and message are two fields. The issue asked for "an error flag".
One field cannot be both a boolean a consumer branches on and a carrier for
the provider's message, so result has "error": <bool> alongside
"message": <string|null>. On AgentError::Truncated, text carries the
partial rather than being blanked.

--bg --json prints one dispatched object ({"kind":"dispatched", "session_id":…,"pid":…}) in place of the dispatched <id> line. A detached
run has no stream and no completion, so a terminal object would be a lie. The
flag is not forwarded into the child, for the same reason -v is not: the
child's stdout and stderr share one log file, so NDJSON there would interleave
with prose.

Errors still exit nonzero with an anyhow line on stderr. The stream gets
its terminal object so completion is never inferred from EOF, but the exit
code remains the authority — extending the precedent set by sessions --json
in #6 rather than contradicting it.

Shape follows sessions --json: a module of pure -> serde_json::Value
builders owned by the CLI, hand-built with json!, unit-tested without
touching disk. run() picks between the existing human printer and a JSON
writer that does write_all + flush per line on a single stdout handle, so
line order is the order the loop produced events in.

Verification

All three CI gates clean on the pinned toolchain (rustc 1.97.1):
cargo fmt --all --check, cargo clippy --workspace --all-targets -D warnings, cargo test --workspace — 108 passed, 0 failed, 10 new (the
bullpen crate goes 4 → 14). Clippy was re-run after touching the three
changed source files, since the first invocation finished off cache.

Manual, against the built binary with BULLPEN_HOME pointed at a scratch dir
and -p openrouter:

Check	Result
Every stdout line parses as JSON	5/5 lines, 0 bad
Flushed mid-flight	tool_start at +3s, result at +10s, across a 6s sleep in a bash call
--bg --json	exactly 1 line, {"kind":"dispatched","pid":…,"session_id":<full uuid>}
--bg without --json	dispatched a11a38f9 (pid 84880) — unchanged
Error path (-m no/such-model)	stdout = one result with "error":true + provider message; stderr = the anyhow line; exit 1
Unflagged run -v	stdout is exactly hello\n (od-verified); footer and tool prose on stderr

Known gap

No test executes the binary, so "stdout is NDJSON and nothing else" and the
byte-for-byte-unchanged unflagged path rest on the diff shape (a bool guard
around existing println!s) plus the manual runs above. Closing that would
add the workspace's first [[test]] target and [dev-dependencies] on
crates/cli — a new convention, and the same call #6 made.

Summary by CodeRabbit

• New Features
  • Added bullpen run --json for newline-delimited JSON event streaming.
  • JSON output includes agent activity, tool metadata, usage, session IDs, errors, and final results.
  • Background dispatch events are included in the JSON stream.
• Improvements
  • Tool results are capped at 256 KiB, with safe UTF-8 truncation.
  • Human-readable output remains unchanged when JSON mode is not enabled.
• Documentation
  • Updated architecture and quickstart documentation with JSON streaming details.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b38e4674-e5b8-459e-b075-d11c345898da

📥 Commits

Reviewing files that changed from the base of the PR and between 7dbb753 and ce29682.

📒 Files selected for processing (5)

• ARCHITECTURE.md
• README.md
• crates/agent/src/lib.rs
• crates/cli/src/json.rs
• crates/cli/src/main.rs

---

📝 Walkthrough

Walkthrough

The CLI adds bullpen run --json with stable NDJSON event names, capped tool payloads, streamed event output, background dispatch records, and a terminal result object. Human-readable output remains unchanged when JSON mode is disabled.

Changes

NDJSON event streaming

Layer / File(s)	Summary
Wire format and size limits crates/agent/src/lib.rs, crates/cli/src/json.rs, ARCHITECTURE.md	The CLI defines stable event serializers for agent events, results, usage, errors, and dispatch metadata. Tool data is capped at 256 KiB. Output is truncated at UTF-8 boundaries and emitted as flushed JSON lines.
CLI JSON execution flow crates/cli/src/main.rs, README.md	bullpen run accepts --json. JSON mode emits dispatch and agent events, suppresses raw assistant text, and emits a terminal result with session ID, output, usage, and errors. The Quickstart documents the command.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Run as bullpen run --json
  participant Consumer as CLI event consumer
  participant Serializer as cli::json serializers
  participant Stdout
  Run->>Consumer: enable JSON event handling
  Consumer->>Serializer: serialize agent event
  Serializer->>Stdout: emit one flushed JSON line
  Run->>Serializer: serialize terminal result
  Serializer->>Stdout: emit result JSON line

Loading

Possibly related issues

• #5 — The PR adds machine-readable JSON output through --json, but it targets bullpen run and uses a separate serialization path.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/run-json-stream

---

_{Comment @coderabbitai help to get the list of available commands.}

[Steel-tech]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[Steel-tech]

@coderabbitai review

[coderabbitai]

⚠️ Action not completed

Pull request is closed.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.