Agent Blackbox

A local flight recorder, verifier, and replay lab for coding agents.

Agent Blackbox launches Codex, OpenCode, Hermes, or a custom command inside a disposable Git worktree. It records a SHA-256 hash-chained event log, runs the repository's verification contract, preserves the resulting patch, and exports a redacted receipt.

No account. No cloud service. No runtime dependency outside Python 3.11+ and Git.

What works in this MVP

• Codex, OpenCode, Hermes, and custom command adapters
• Native Windows, macOS, and Linux process execution
• Disposable Git worktrees per run
• Streaming stdout/stderr and JSONL event capture
• Best-effort token and cost normalization from agent JSON events
• SQLite run history
• Tamper-evident SHA-256 event chains
• Repository-owned verification commands
• Binary-capable Git patch preservation
• Redacted JSON receipt export with a fresh portable integrity hash
• Replay of the same task across multiple agents
• Promotion of a successful run into a Hermes/Agent Skills-style SKILL.md
• Dependency-free localhost dashboard

Install

From the extracted repository:

py -3.11 -m pip install -e .
blackbox --version

Or run directly without installing:

py -3.11 blackbox.py --help

Start in a repository

cd C:\code\my-project
blackbox init

Edit blackbox.toml and add the checks that define success for that repository:

[verify]
commands = [
  "python -m pytest -q",
]

The default adapters are intentionally visible and editable:

[adapters.codex]
command = ["codex", "exec", "--json", "--full-auto", "{task}"]

[adapters.opencode]
command = ["opencode", "run", "--format", "json", "{task}"]

[adapters.hermes]
command = ["hermes", "--yolo", "chat", "-q", "{task}"]

These automation modes allow agents to execute commands. Blackbox isolates repository changes in a worktree, but it is not an operating-system sandbox. Use the permission and sandbox controls provided by each agent.

Record a run

blackbox run codex --task "Fix the invoice PDF table overflow"

Override or add verification from the command line:

blackbox run hermes `
  --task "Fix the parser and add a regression test" `
  --verify "python -m unittest discover -s tests -v"

Run any local agent or script through the custom adapter:

blackbox run custom `
  --task "Update the changelog" `
  --command 'my-agent --prompt "{task}" --cwd "{workspace}"'

The disposable worktree is removed after the run. Its patch remains under:

.agent-blackbox/artifacts/<run-id>.patch

Apply it to your current checkout after review:

git apply .agent-blackbox\artifacts\<run-id>.patch

Use --keep-worktree to inspect the complete workspace instead.

Compare agents

blackbox replay <run-id> --agents codex,opencode,hermes

Each replay starts from the same committed revision and uses the same task and verification contract.

Inspect and export receipts

blackbox list
blackbox show <run-id> --events
blackbox verify-chain <run-id>
blackbox export <run-id>
blackbox verify-receipt .agent-blackbox/receipts/<run-id>.json

Exports are redacted and written to .agent-blackbox/receipts/ by default.

Promote a run into a skill

blackbox promote <run-id> --name fix-pdf-table-overflow

This creates:

.agent-blackbox/skills/fix-pdf-table-overflow/SKILL.md

The generated skill is deliberately conservative. It captures the task, touched files, verification contract, and receipt provenance without pretending that one successful trajectory is universally correct. Review it before reuse.

Dashboard

blackbox serve

Open http://127.0.0.1:7417.

The dashboard reads the same local SQLite database. It does not expose mutation endpoints and binds to localhost by default.

Doctor

blackbox doctor

This checks Python, Git, the repository, and the three built-in agent executables.

Development

py -3.11 -m unittest discover -s tests -v

The MVP keeps runtime dependencies at zero so it remains easy to inspect, hack, and carry between repositories.

Current limits

• Token and dollar-cost extraction depends on what each agent emits. Common usage keys are normalized, while unknown provider-specific shapes remain available in raw JSONL events.
• A Git worktree starts from committed HEAD; uncommitted edits in the source checkout are not copied into the replay environment.
• Agent Blackbox isolates Git changes, not arbitrary filesystem or network access.
• Replay runs agents sequentially in this version.
• Patch application remains a deliberate human step.

License

MIT