Phase 3: Trade journal + Claude behavioral-analysis agent

.env.example

@@ -0,0 +1,10 @@
+# Copy to .env and fill in. The behavioral-analysis agent (Phase 3) needs a key.
+
+# Required for `POST /journal/analyze` and `python -m backtest.coach`.
+ANTHROPIC_API_KEY=
+
+# Optional. Defaults to claude-opus-4-8.
+# ANTHROPIC_MODEL=claude-opus-4-8
+
+# Optional. Path to the SQLite database (metrics + journal). Defaults to metrics.db.
+# METRICS_DB=metrics.db

.gitignore

@@ -18,6 +18,9 @@ out/
 *.db
 *.sqlite3
 
+# Secrets (Claude API key etc.)
+.env
+
 # Editor / OS
 .vscode/
 .idea/

README.md

@@ -36,7 +36,7 @@ Early scaffolding. Roadmap:
 - [x] Phase 0 — Scaffolding: monorepo layout, linters, CI.
 - [x] Phase 1 — Vertical slice: sample tick CSV → order-flow imbalance → SQLite → API endpoint.
 - [x] Phase 2 — More metrics (realized volatility, volume, VWAP), tests against known data.
-- [ ] Phase 3 — Trade journal model + Claude API behavioral agent.
+- [x] Phase 3 — Trade journal model + Claude API behavioral agent.
 - [ ] Phase 4 — Next.js dashboard.
 - [ ] Phase 5 — v0.1.0 release with sample-data demo.
 
@@ -63,6 +63,28 @@ curl 'localhost:8000/metrics/volatility?limit=5'  # realized volatility + trade
 curl 'localhost:8000/metrics/volume?limit=5'      # buy/sell/total volume + VWAP
 ```
 
+Phase 3 adds a trade journal (stored alongside the metrics) and a Claude-powered
+behavioral agent that reads it. The agent needs an Anthropic API key; it produces
+behavioral observations correlated with the market regime — **not** trading advice
+or price predictions.
+
+```bash
+# 5. Generate a synthetic trade journal (deterministic; all synthetic)
+python3 data/generate_journal.py                  # writes data/sample_journal.csv
+
+# 6. Analyze behavior with Claude (defaults to claude-opus-4-8;
+#    override with ANTHROPIC_MODEL). --load seeds the journal into the DB first.
+export ANTHROPIC_API_KEY=sk-ant-...
+python3 -m backtest.coach --load data/sample_journal.csv --db metrics.db
+
+# 7. Or drive the journal over the API (shares the same METRICS_DB)
+curl -X POST localhost:8000/journal -H 'content-type: application/json' \
+  -d '{"symbol":"MNQ","side":"long","entered_at_ns":1760000030000000000,
+       "entry_price":21400,"size":1,"notes":"followed my plan","emotion":"calm"}'
+curl 'localhost:8000/journal?limit=5'             # entries joined to the market regime
+curl -X POST 'localhost:8000/journal/analyze'     # Claude behavioral analysis (503 without a key)
+```
+
 Run the checks the same way CI does:
 
 ```bash
@@ -112,3 +134,16 @@ calls were made by me — as the project progresses.
 - **VWAP folded into the same pass** (Phase 2, Claude's proposal, accepted): it needs
   only the running `Σ price·size` the engine already touches per tick, so it comes almost
   for free and gives the dashboard a price anchor alongside the volume figures.
+- **The trade journal is Python-owned and lives in the same database** (Phase 3, mine):
+  unlike `metrics`, the journal is written and read by the Python + agent layer, not the
+  Rust hot path, so Python owns the `journal_entries` schema — a deliberate inversion of
+  the Phase 1 "the engine owns the schema" rule. Keeping it in the same SQLite file lets
+  each entry join to the microstructure regime at its entry time (`GET /journal` and the
+  agent both use that join), which is the whole point of the feature. SQLite only, no ORM,
+  to stay consistent with the rest of the stack; Postgres in the diagram stays a future
+  option, not a Phase 3 dependency.
+- **Behavioral agent defaults to `claude-opus-4-8`** (Phase 3, mine): overridable via the
+  `ANTHROPIC_MODEL` env var. It emits behavioral observations tied to the regime — by
+  design **not** trading advice and **not** price predictions. The Anthropic client is
+  dependency-injected, so the analysis is unit-tested against a mock and CI needs neither
+  a key nor network access.

backtest/pyproject.toml

@@ -10,6 +10,7 @@ requires-python = ">=3.11"
 dependencies = [
     "fastapi>=0.110",
     "uvicorn>=0.29",
+    "anthropic>=0.117",
 ]
 
 [project.optional-dependencies]

backtest/src/backtest/api.py

@@ -1,22 +1,59 @@
-"""Read-only API over the metrics database produced by the Rust engine.
-
-The engine owns the schema; this layer only exposes it. Point it at a
-database with the METRICS_DB environment variable (default: metrics.db
-in the working directory).
+"""API over the metrics database and the trade journal.
+
+The metrics endpoints are read-only: the Rust engine owns that schema and this
+layer only exposes it. The journal endpoints (Phase 3) are read/write — the
+journal is owned by this Python layer and stored alongside the metrics in the
+same database, so entries can be joined to the market regime at entry time. Point
+the app at a database with the METRICS_DB environment variable (default:
+metrics.db in the working directory).
 """
 
 import os
 import sqlite3
 from pathlib import Path
-from typing import Any
+from typing import Annotated, Any
+
+from fastapi import Depends, FastAPI, HTTPException, Query
+from pydantic import BaseModel
 
-from fastapi import FastAPI, HTTPException, Query
+from backtest import coach, journal
 
 DB_PATH_ENV = "METRICS_DB"
 
 app = FastAPI(title="Trading Microstructure Engine API")
 
 
+class JournalEntryIn(BaseModel):
+    """A trade the user is logging to their journal."""
+
+    symbol: str
+    side: str
+    entered_at_ns: int
+    entry_price: float
+    size: int
+    exited_at_ns: int | None = None
+    exit_price: float | None = None
+    pnl: float | None = None
+    notes: str | None = None
+    emotion: str | None = None
+
+
+def get_anthropic_client() -> Any:
+    """Provide an Anthropic client, or 503 when no API key is configured.
+
+    Overridable in tests via ``app.dependency_overrides`` so the behavioral
+    endpoint can be exercised without a real key or network access.
+    """
+    if not os.environ.get("ANTHROPIC_API_KEY"):
+        raise HTTPException(
+            status_code=503,
+            detail="ANTHROPIC_API_KEY is not set — the behavioral agent is unavailable",
+        )
+    import anthropic
+
+    return anthropic.Anthropic()
+
+
 def _db_path() -> Path:
     return Path(os.environ.get(DB_PATH_ENV, "metrics.db"))
 
@@ -67,3 +104,30 @@ def get_volume(limit: int = Query(default=100, ge=1, le=10_000)) -> list[dict[st
         "bucket_start_ns, window_ns, buy_volume, sell_volume, total_volume, vwap, trade_count",
         limit,
     )
+
+
+@app.post("/journal", status_code=201)
+def create_journal_entry(entry: JournalEntryIn) -> dict[str, Any]:
+    """Log a trade to the journal, returning the stored row (with its id)."""
+    return journal.insert_entry(_db_path(), entry.model_dump())
+
+
+@app.get("/journal")
+def get_journal(limit: int = Query(default=100, ge=1, le=10_000)) -> list[dict[str, Any]]:
+    """Return journal entries newest first, each joined to its market regime.
+
+    An empty journal (or a database that does not exist yet) returns ``[]`` —
+    unlike the metrics endpoints, the journal is created by this layer, so
+    "no entries yet" is a normal state rather than a 503.
+    """
+    return journal.list_enriched(_db_path(), limit)
+
+
+@app.post("/journal/analyze")
+def analyze_journal(
+    client: Annotated[Any, Depends(get_anthropic_client)],
+    limit: int = Query(default=100, ge=1, le=10_000),
+) -> coach.BehavioralAnalysis:
+    """Run the Claude behavioral-analysis agent over the journal."""
+    entries = journal.list_enriched(_db_path(), limit)
+    return coach.analyze(entries, client=client)

backtest/src/backtest/coach.py

@@ -0,0 +1,189 @@
+"""Claude-powered behavioral-analysis agent (the "coach").
+
+Reads the trade journal joined to the market microstructure regime at each entry
+(see :mod:`backtest.journal`) and asks Claude to surface recurring behavioral
+patterns and emotional biases. It produces *behavioral observations about the
+trader's own decisions* — never trading advice and never price predictions.
+
+The Anthropic client is injected into :func:`analyze` so the analysis logic is
+unit-testable with a fake client and no network access. The SDK is imported
+lazily in :func:`main` (the CLI) so importing this module does not require the
+``anthropic`` package.
+
+CLI:
+    python -m backtest.coach [--db PATH] [--limit N] [--load CSV]
+"""
+
+import argparse
+import os
+from typing import Any, Literal
+
+from pydantic import BaseModel, Field
+
+from backtest import journal
+
+MODEL_ENV = "ANTHROPIC_MODEL"
+DEFAULT_MODEL = "claude-opus-4-8"
+# Same env var the read API uses (backtest.api.DB_PATH_ENV); duplicated here to
+# avoid a circular import, since api imports coach.
+DB_PATH_ENV = "METRICS_DB"
+
+DISCLAIMER = (
+    "This is a behavioral analysis of your own trading. It is not financial "
+    "advice and does not predict prices or market direction."
+)
+
+SYSTEM_PROMPT = f"""\
+You are a trading-behavior analyst. You read a trader's own journal — the trades
+they chose to take, their written notes and stated emotions — alongside the
+market microstructure regime (order-flow imbalance, realized volatility, VWAP)
+that was in force at the moment of entry.
+
+Your job is to surface recurring behavioral patterns and emotional or cognitive
+biases, and to relate them to the regime the trader was acting in — for example
+entering into strongly one-sided order flow, sizing up right after a loss, or
+hesitating during high-volatility windows. Ground every observation in specific
+journal entries.
+
+You do NOT give trading advice, you do NOT tell the trader what to do next, and
+you do NOT predict prices or market direction. You describe behavior, not
+markets.
+
+Populate the 'disclaimer' field with exactly this text: "{DISCLAIMER}"
+"""
+
+
+class BehavioralObservation(BaseModel):
+    """One recurring behavioral pattern found in the journal."""
+
+    pattern: str = Field(description="Short name for the behavioral pattern.")
+    bias: str = Field(description="The emotional or cognitive bias involved.")
+    evidence: str = Field(description="Specific journal entries that support it.")
+    severity: Literal["low", "medium", "high"] = Field(
+        description="How strongly the pattern shows up in the journal."
+    )
+
+
+class BehavioralAnalysis(BaseModel):
+    """The agent's structured read of the whole journal."""
+
+    summary: str = Field(description="One-paragraph overview of the trader's behavior.")
+    observations: list[BehavioralObservation] = Field(
+        description="Distinct behavioral patterns, most notable first."
+    )
+    disclaimer: str = Field(description="Verbatim behavioral-analysis disclaimer.")
+
+
+def _model() -> str:
+    return os.environ.get(MODEL_ENV, DEFAULT_MODEL)
+
+
+def build_messages(entries: list[dict[str, Any]]) -> tuple[str, list[dict[str, Any]]]:
+    """Build the (system, messages) pair sent to Claude for ``entries``."""
+    lines = [
+        "Here is the trader's own journal. Each entry pairs a trade they took "
+        "with the market microstructure regime at the moment they entered it.",
+        "",
+    ]
+    lines.extend(_format_entry(entry) for entry in entries)
+    lines.extend(
+        [
+            "",
+            "Analyze this journal for recurring behavioral patterns and "
+            "emotional biases, and how they relate to the regime at entry. Cite "
+            "specific entries as evidence. Do not give trading advice or predict "
+            "prices.",
+        ]
+    )
+    return SYSTEM_PROMPT, [{"role": "user", "content": "\n".join(lines)}]
+
+
+def analyze(
+    entries: list[dict[str, Any]],
+    *,
+    client: Any,
+    model: str | None = None,
+) -> BehavioralAnalysis:
+    """Return Claude's structured behavioral analysis of ``entries``.
+
+    ``client`` is an Anthropic client (or any object exposing the same
+    ``messages.parse`` surface), injected so this is testable without network
+    access. An empty journal short-circuits to an empty analysis rather than
+    spending an API call.
+    """
+    if not entries:
+        return BehavioralAnalysis(
+            summary="No journal entries to analyze yet.",
+            observations=[],
+            disclaimer=DISCLAIMER,
+        )
+    system, messages = build_messages(entries)
+    response = client.messages.parse(
+        model=model or _model(),
+        max_tokens=16000,
+        thinking={"type": "adaptive"},
+        system=system,
+        messages=messages,
+        output_format=BehavioralAnalysis,
+    )
+    return response.parsed_output
+
+
+def _format_entry(entry: dict[str, Any]) -> str:
+    return (
+        f"#{entry.get('id', '?')} {entry.get('symbol')} {entry.get('side')} "
+        f"size={entry.get('size')} entry={entry.get('entry_price')} "
+        f"exit={entry.get('exit_price')} pnl={entry.get('pnl')} "
+        f"entered_at_ns={entry.get('entered_at_ns')} | "
+        f"regime: {_format_regime(entry)} | "
+        f"emotion={entry.get('emotion')!r} notes={entry.get('notes')!r}"
+    )
+
+
+def _format_regime(entry: dict[str, Any]) -> str:
+    if entry.get("regime_ofi") is None:
+        return "unknown (no metrics for this window)"
+    return (
+        f"ofi={entry.get('regime_ofi')} "
+        f"realized_volatility={entry.get('regime_realized_volatility')} "
+        f"vwap={entry.get('regime_vwap')}"
+    )
+
+
+def _render(analysis: BehavioralAnalysis) -> str:
+    lines = [analysis.summary, ""]
+    for obs in analysis.observations:
+        lines.append(f"- [{obs.severity}] {obs.pattern} ({obs.bias})")
+        lines.append(f"    {obs.evidence}")
+    lines.extend(["", analysis.disclaimer])
+    return "\n".join(lines)
+
+
+def main(argv: list[str] | None = None) -> None:
+    parser = argparse.ArgumentParser(description="Analyze a trade journal with Claude.")
+    parser.add_argument("--db", default=os.environ.get(DB_PATH_ENV, "metrics.db"))
+    parser.add_argument("--limit", type=int, default=100)
+    parser.add_argument(
+        "--load",
+        metavar="CSV",
+        help="seed the journal from a CSV before analyzing",
+    )
+    args = parser.parse_args(argv)
+
+    if args.load:
+        count = journal.import_csv(args.load, args.db)
+        print(f"loaded {count} journal entries from {args.load}")
+
+    entries = journal.list_enriched(args.db, args.limit)
+    if not entries:
+        print("no journal entries found; add trades or pass --load <csv>")
+        return
+
+    import anthropic  # lazy: keeps the SDK out of the library import path
+
+    analysis = analyze(entries, client=anthropic.Anthropic())
+    print(_render(analysis))
+
+
+if __name__ == "__main__":
+    main()