ed-donner · VincentKovar · Jul 20, 2026
diff --git a/planning/MARKET_INTERFACE.md b/planning/MARKET_INTERFACE.md
@@ -0,0 +1,132 @@
+# Market Data Interface — Unified Design
+
+Defines the single abstract interface (`MarketDataSource`, per PLAN.md §6) that both the Massive API client and the built-in simulator implement, so that price-cache and SSE code (and everything downstream) is agnostic to the source. Backed by research in `MASSIVE_API.md`; simulator internals are detailed in `MARKET_SIMULATOR.md`.
+
+## 1. Selection Rule
+
+```python
+def build_market_data_source() -> "MarketDataSource":
+    if os.environ.get("MASSIVE_API_KEY"):
+        return MassiveMarketDataSource(api_key=os.environ["MASSIVE_API_KEY"])
+    return SimulatorMarketDataSource()
+```
+
+- `MASSIVE_API_KEY` set and non-empty → `MassiveMarketDataSource`
+- Absent/empty → `SimulatorMarketDataSource` (default, no external dependency)
+
+This selection happens once at process startup; the chosen source runs as the single background task described in PLAN.md §6 ("Shared Price Cache").
+
+## 2. Shared Data Model
+
+```python
+from dataclasses import dataclass
+
+@dataclass(frozen=True)
+class PriceTick:
+    ticker: str
+    price: float          # latest trade price
+    previous_close: float # prior session close, for daily % change
+    timestamp: float       # unix seconds (epoch), UTC
+```
+
+Both implementations produce `PriceTick` objects in this exact shape — this is the contract the price cache and SSE stream depend on. Neither the simulator's GBM internals nor Massive's snapshot JSON shape leak past this boundary.
+
+## 3. Abstract Interface
+
+```python
+from abc import ABC, abstractmethod
+from typing import Iterable
+
+class MarketDataSource(ABC):
+    @abstractmethod
+    async def start(self, tickers: Iterable[str]) -> None:
+        """Begin producing price updates for the given tickers (initial watchlist)."""
+
+    @abstractmethod
+    async def stop(self) -> None:
+        """Stop the background update loop and release any resources."""
+
+    @abstractmethod
+    def watch(self, ticker: str) -> None:
+        """Add a ticker to the set of symbols being updated (e.g. added to watchlist, or a position opened on a delisted-from-watchlist ticker)."""
+
+    @abstractmethod
+    def unwatch(self, ticker: str) -> None:
+        """Stop updating a ticker once nothing references it (removed from watchlist and no open position)."""
+
+    @abstractmethod
+    def latest(self, ticker: str) -> PriceTick | None:
+        """Return the most recent PriceTick for a ticker, or None if not yet available."""
+
+    @abstractmethod
+    def subscribe(self) -> "AsyncIterator[PriceTick]":
+        """Async stream of PriceTick updates as they occur, for the SSE endpoint to consume."""
+```
+
+Both `MassiveMarketDataSource` and `SimulatorMarketDataSource` (§4/§5 below) implement this exact interface.
+
+## 4. `MassiveMarketDataSource`
+
+- `start()` launches an `asyncio` polling loop calling the Massive **Full Market Snapshot** endpoint (`MASSIVE_API.md` §3.1) once per tick with the comma-joined set of currently-watched tickers.
+- Poll interval: env-configurable (`MASSIVE_POLL_INTERVAL_SECONDS`, default `15` to respect the free-tier 5 req/min limit; lower for paid tiers).
+- Each response is parsed into one `PriceTick` per ticker: `price = lastTrade.p`, `previous_close = prevDay.c`, `timestamp = updated / 1e9` (Massive returns nanoseconds).
+- `watch()`/`unwatch()` mutate the in-memory ticker set consulted on the next tick — no immediate network call, to stay within rate limits. (Exception: consider an immediate single-ticker snapshot call, `MASSIVE_API.md` §3.2, when a brand-new ticker is added, so its first price isn't delayed a full poll interval — flagged here as a possible enhancement, not required for v1.)
+- On HTTP error or 429: log and skip the tick, keep serving the last cached `PriceTick` per ticker (never fall back to the simulator mid-session — see `MASSIVE_API.md` §4).
+- Unknown/invalid tickers: if a requested ticker is absent from the response, mark it `None` in `latest()` rather than raising.
+
+```python
+class MassiveMarketDataSource(MarketDataSource):
+    def __init__(self, api_key: str, poll_interval: float = 15.0):
+        self._api_key = api_key
+        self._poll_interval = poll_interval
+        self._tickers: set[str] = set()
+        self._cache: dict[str, PriceTick] = {}
+        self._subscribers: list[asyncio.Queue[PriceTick]] = []
+        self._task: asyncio.Task | None = None
+
+    async def start(self, tickers):
+        self._tickers = set(tickers)
+        self._task = asyncio.create_task(self._poll_loop())
+
+    async def _poll_loop(self):
+        async with httpx.AsyncClient(base_url="https://api.massive.com") as client:
+            while True:
+                if self._tickers:
+                    await self._poll_once(client)
+                await asyncio.sleep(self._poll_interval)
+
+    async def _poll_once(self, client):
+        try:
+            resp = await client.get(
+                "/v2/snapshot/locale/us/markets/stocks/tickers",
+                params={"tickers": ",".join(sorted(self._tickers))},
+                headers={"Authorization": f"Bearer {self._api_key}"},
+            )
+            resp.raise_for_status()
+        except httpx.HTTPError:
+            return  # skip this tick, keep last known prices
+        for entry in resp.json().get("tickers", []):
+            tick = PriceTick(
+                ticker=entry["ticker"],
+                price=entry["lastTrade"]["p"],
+                previous_close=entry["prevDay"]["c"],
+                timestamp=entry["updated"] / 1e9,
+            )
+            self._cache[tick.ticker] = tick
+            self._publish(tick)
+```
+
+## 5. `SimulatorMarketDataSource`
+
+- `start()` launches an `asyncio` loop ticking every ~500ms (PLAN.md §6), generating `PriceTick`s via GBM per `MARKET_SIMULATOR.md`.
+- `watch()`/`unwatch()` add/remove a ticker from the simulated set immediately, seeding new tickers from `SEED_PRICES` (or a synthesized default if unrecognized — see `MARKET_SIMULATOR.md` §5).
+- No network calls, no rate limits, no failure mode to handle.
+
+## 6. Price Cache & SSE Integration
+
+- Whichever `MarketDataSource` is active, the same cache/SSE code in the backend calls `latest()` for REST responses (`/api/watchlist`, `/api/portfolio`) and `subscribe()` to fan out `PriceTick`s to connected `/api/stream/prices` clients.
+- SSE event payload is `PriceTick` serialized to JSON plus a derived `direction` field (`"up" | "down" | "flat"`, computed by comparing to the previously cached price before overwriting).
+
+## 7. Watchlist / Position Lifecycle Hooks
+
+Per the open question in `planning/REVIEW.md` about deleting a watched ticker with an open position: the backend calls `source.watch(ticker)` whenever a ticker is added to the watchlist **or** a position is opened, and `source.unwatch(ticker)` only when a ticker is both off the watchlist **and** has zero position quantity. This keeps the `MarketDataSource` implementations simple (they just track a flat ticker set) while the lifecycle policy lives in the backend layer that calls them.
diff --git a/planning/MARKET_SIMULATOR.md b/planning/MARKET_SIMULATOR.md
@@ -0,0 +1,130 @@
+# Market Simulator — Design
+
+Design for `SimulatorMarketDataSource` (PLAN.md §6, `MARKET_INTERFACE.md` §5), the default no-API-key market data source. Produces plausible, continuously-updating stock prices without any external dependency.
+
+## 1. Goals
+
+- Look and feel like a real, moving market: correlated sector moves, occasional volatility spikes, no external calls.
+- Deterministic enough to unit test (seedable RNG), realistic enough for a convincing demo.
+- Conform exactly to the `MarketDataSource` interface (`MARKET_INTERFACE.md` §3) so the rest of the system can't tell it apart from the Massive-backed source.
+
+## 2. Price Model — Geometric Brownian Motion (GBM)
+
+Each ticker's price evolves per discrete-time GBM:
+
+```
+S(t + dt) = S(t) * exp((mu - 0.5 * sigma^2) * dt + sigma * sqrt(dt) * Z)
+```
+
+- `S(t)`: current price
+- `mu`: drift (annualized), per-ticker, small (e.g. `0.05` = 5%/yr) so prices don't trend away over a demo session
+- `sigma`: annualized volatility, per-ticker (e.g. `0.20`–`0.45`, higher for TSLA/NVDA-style names)
+- `dt`: elapsed time as a fraction of a trading year for one tick (`tick_interval_seconds / (252 * 6.5 * 3600)`, i.e. treating each tick as a fraction of a 6.5-hour trading day)
+- `Z`: standard normal random draw, independent per tick but **correlated across tickers** (§3)
+
+## 3. Correlated Moves
+
+To make sector co-movement visible (PLAN.md: "tech stocks move together"), tickers are grouped, and each tick draws:
+
+1. One shared market-wide factor `Z_market ~ N(0,1)`
+2. One sector factor `Z_sector ~ N(0,1)` per group (e.g. `tech`, `finance`)
+3. One idiosyncratic factor `Z_idio ~ N(0,1)` per ticker
+
+Each ticker's `Z` is a weighted blend:
+
+```
+Z_ticker = beta_market * Z_market + beta_sector * Z_sector + beta_idio * Z_idio
+```
+
+with `beta_market + beta_sector + beta_idio` chosen so `Z_ticker` stays unit-variance (e.g. weights `0.4, 0.3, sqrt(1 - 0.4^2 - 0.3^2)`). Default sector groups for the seed watchlist:
+
+| Sector | Tickers |
+|---|---|
+| tech | AAPL, GOOGL, MSFT, AMZN, NVDA, META, NFLX |
+| finance | JPM, V |
+| auto/other | TSLA |
+
+## 4. Random Events
+
+Each tick, per ticker, with small probability (e.g. `p_event = 0.001` at a 500ms tick → roughly one event per ~8 minutes per ticker), apply an independent shock:
+
+```
+S(t) *= (1 + sign * random.uniform(0.02, 0.05))  # sign = +1 or -1, 50/50
+```
+
+This produces the "sudden 2–5% moves... for drama" called for in PLAN.md §6, layered on top of (not replacing) the GBM tick for that step.
+
+## 5. Seed Prices
+
+A single `SEED_PRICES: dict[str, float]` module-level map is the one place starting prices live (addresses the simplification flagged in `planning/REVIEW.md`):
+
+```python
+SEED_PRICES: dict[str, float] = {
+    "AAPL": 190.00, "GOOGL": 175.00, "MSFT": 420.00, "AMZN": 185.00,
+    "TSLA": 245.00, "NVDA": 135.00, "META": 590.00, "JPM": 210.00,
+    "V": 275.00, "NFLX": 680.00,
+}
+DEFAULT_SEED_PRICE = 100.00  # used for any ticker not in the map (e.g. user/AI adds an unrecognized symbol)
+```
+
+When `watch(ticker)` is called for a ticker not already tracked, its starting price is `SEED_PRICES.get(ticker, DEFAULT_SEED_PRICE)`, and it's assigned to the `misc` sector group (`beta_sector = 0`, i.e. only market + idiosyncratic factors) unless explicitly added to a sector table.
+
+## 6. Tick Loop
+
+```python
+class SimulatorMarketDataSource(MarketDataSource):
+    def __init__(self, tick_interval: float = 0.5, seed: int | None = None):
+        self._tick_interval = tick_interval
+        self._rng = random.Random(seed)
+        self._prices: dict[str, float] = {}
+        self._sectors: dict[str, str] = {}
+        self._cache: dict[str, PriceTick] = {}
+        self._session_open: dict[str, float] = {}  # previous_close per ticker
+        self._subscribers: list[asyncio.Queue[PriceTick]] = []
+        self._task: asyncio.Task | None = None
+
+    async def start(self, tickers):
+        for t in tickers:
+            self.watch(t)
+        self._task = asyncio.create_task(self._tick_loop())
+
+    def watch(self, ticker: str) -> None:
+        if ticker in self._prices:
+            return
+        price = SEED_PRICES.get(ticker, DEFAULT_SEED_PRICE)
+        self._prices[ticker] = price
+        self._session_open[ticker] = price
+        self._sectors[ticker] = _sector_for(ticker)
+
+    async def _tick_loop(self):
+        while True:
+            self._advance_all()
+            await asyncio.sleep(self._tick_interval)
+
+    def _advance_all(self):
+        z_market = self._rng.gauss(0, 1)
+        z_sector = {s: self._rng.gauss(0, 1) for s in set(self._sectors.values())}
+        for ticker, price in self._prices.items():
+            z = _blend(z_market, z_sector[self._sectors[ticker]], self._rng.gauss(0, 1))
+            new_price = _gbm_step(price, mu=0.05, sigma=_sigma_for(ticker), dt=_DT, z=z)
+            if self._rng.random() < 0.001:
+                sign = self._rng.choice([-1, 1])
+                new_price *= 1 + sign * self._rng.uniform(0.02, 0.05)
+            self._prices[ticker] = new_price
+            tick = PriceTick(
+                ticker=ticker,
+                price=round(new_price, 4),
+                previous_close=self._session_open[ticker],
+                timestamp=time.time(),
+            )
+            self._cache[ticker] = tick
+            self._publish(tick)
+```
+
+`_gbm_step` and `_blend` implement §2/§3 directly; `_DT` is the per-tick fraction of a trading year derived from `tick_interval`.
+
+## 7. Testability
+
+- `seed` param makes the RNG deterministic for unit tests (`GBM math is correct`, per PLAN.md §12).
+- Because `_advance_all` is a pure step function over `self._prices`, tests can call it directly N times and assert statistical properties (mean drift ≈ `mu`, no negative prices, event shocks bounded to 2–5%) without needing to run the async tick loop.
+- Conformance test: a shared test suite (parametrized over both `SimulatorMarketDataSource` and a mocked `MassiveMarketDataSource`) asserts both satisfy the `MarketDataSource` ABC contract from `MARKET_INTERFACE.md` §3.
diff --git a/planning/MASSIVE_API.md b/planning/MASSIVE_API.md
@@ -0,0 +1,106 @@
+# Massive API (formerly Polygon.io) — Reference for FinAlly
+
+Research notes and code examples for the endpoints FinAlly's market-data layer needs: real-time/latest quotes for multiple tickers and end-of-day (previous close) prices. This document is the input to `MARKET_INTERFACE.md`, which designs the unified Python interface built on top of it.
+
+Sources: [API Docs](https://massive.com/docs), [Stocks REST Overview](https://massive.com/docs/rest/stocks/overview), [Full Market Snapshot](https://massive.com/docs/rest/stocks/snapshots/full-market-snapshot.md), [Unified Snapshot](https://massive.com/docs/rest/stocks/snapshots/unified-snapshot.md), [Single Ticker Snapshot](https://massive.com/docs/rest/stocks/snapshots/single-ticker-snapshot.md), [Previous Day Bar](https://massive.com/docs/rest/stocks/aggregates/previous-day-bar.md), [Rate Limits](https://massive.com/knowledge-base/article/what-is-the-request-limit-for-massives-restful-apis), [Python client](https://github.com/massive-com/client-python)
+
+> Polygon.io rebranded to **Massive** in early 2026. `MASSIVE_API_KEY` in `.env` refers to this service.
+
+## 1. Base URL & Authentication
+
+- Base URL: `https://api.massive.com`
+- Auth: `Authorization: Bearer <MASSIVE_API_KEY>` HTTP header (the official Python client sends this; the legacy Polygon.io `?apiKey=` query param may still work but the header form is documented as current).
+
+```bash
+curl -H "Authorization: Bearer $MASSIVE_API_KEY" \
+  "https://api.massive.com/v2/snapshot/locale/us/markets/stocks/tickers?tickers=AAPL,GOOGL,MSFT"
+```
+
+## 2. Rate Limits
+
+- **Free tier**: 5 requests/minute.
+- **Paid tiers**: effectively unlimited, but Massive still asks clients to stay under ~100 req/s to avoid throttling.
+- This is the reason PLAN.md §6 specifies polling (not per-request-per-tick) on a 15s interval for the free tier.
+
+## 3. Endpoints Relevant to FinAlly
+
+### 3.1 Full Market Snapshot (multi-ticker, one call) — **primary endpoint for polling the watchlist**
+
+```
+GET /v2/snapshot/locale/us/markets/stocks/tickers?tickers=AAPL,TSLA,GOOG
+```
+
+Params:
+- `tickers` (optional, comma-separated, case-sensitive): restricts the snapshot to specific symbols — this is how FinAlly fetches all watchlist tickers in a single call.
+- `include_otc` (optional, bool, default `false`)
+
+Response:
+
+```json
+{
+  "count": 1,
+  "status": "OK",
+  "tickers": [
+    {
+      "ticker": "AAPL",
+      "todaysChange": -0.124,
+      "todaysChangePerc": -0.601,
+      "day": { "c": 20.506, "h": 20.64, "l": 20.506, "o": 20.64, "v": 37216, "vw": 20.616 },
+      "prevDay": { "c": 20.63, "h": 21, "l": 20.5, "o": 20.79, "v": 292738 },
+      "min": { "c": 20.506, "h": 20.506, "l": 20.506, "o": 20.506, "v": 5000 },
+      "lastQuote": { "P": 20.6, "p": 20.5 },
+      "lastTrade": { "p": 20.506, "s": 2416 },
+      "updated": 1605192894630916600
+    }
+  ]
+}
+```
+
+Fields FinAlly needs: `ticker`, `lastTrade.p` (latest price), `prevDay.c` (previous close, for daily % change fallback), `day.c` (today's close-so-far), `updated` (nanosecond epoch timestamp).
+
+### 3.2 Single Ticker Snapshot
+
+```
+GET /v2/snapshot/locale/us/markets/stocks/tickers/{stocksTicker}
+```
+
+Same shape as one entry of the full snapshot's `tickers[]` array, nested under `ticker` instead. Useful for on-demand lookups (e.g., when the AI or user adds a new ticker to the watchlist and a price is needed immediately, without waiting for the next poll cycle).
+
+### 3.3 Unified Snapshot (multi-asset-class, newer endpoint)
+
+```
+GET /v3/snapshot?ticker.any_of=AAPL,GOOGL,MSFT&limit=10
+```
+
+- `ticker.any_of`: comma-separated list, up to 250 tickers.
+- Returns `last_quote.bid/ask`, `last_trade.price/size`, `session.open/high/low/close/volume`.
+- Broader (covers options/crypto/forex too) but the field names differ from the v2 snapshot. **Recommendation: use 3.1 (`/v2/snapshot/.../tickers`) for FinAlly** since it's stocks-only, matches the schema FinAlly's `MarketDataSource` interface needs (see `MARKET_INTERFACE.md`), and keeps parsing simpler.
+
+### 3.4 Previous Day Bar (end-of-day OHLC)
+
+```
+GET /v2/aggs/ticker/{stocksTicker}/prev?adjusted=true
+```
+
+```json
+{
+  "results": [
+    { "T": "AAPL", "c": 115.97, "h": 117.59, "l": 114.13, "o": 115.55, "t": 1605042000000, "v": 131704427, "vw": 116.3058 }
+  ],
+  "status": "OK",
+  "ticker": "AAPL"
+}
+```
+
+Single-ticker only — for the watchlist's set of tickers this requires one call per ticker, so it's mainly useful for seeding a starting price for a newly-added ticker outside market hours, not for the live polling loop (use 3.1 for that, whose `prevDay` block already carries this data per ticker in one call).
+
+## 4. Recommended Polling Strategy for FinAlly
+
+1. On each poll tick, call **3.1 Full Market Snapshot** once with `tickers=` set to the comma-joined current watchlist (+ any tickers with open positions, per PLAN.md REVIEW.md note on delisted-from-watchlist positions).
+2. Parse each entry into `{ticker, price: lastTrade.p, previous_close: prevDay.c, timestamp: updated}`.
+3. Poll interval: 15s on free tier (5 req/min ceiling — one snapshot call per tick, well under the limit regardless of watchlist size since it's a single request). Configurable via env for paid tiers (down to 2s).
+4. On request failure or rate-limit (HTTP 429): back off and retry next tick; do not fall back to the simulator mid-session (would produce a discontinuous price jump) — just skip the tick and keep the last known price in the cache.
+
+## 5. Ticker Validation
+
+Massive returns `status: "OK"` with an empty or partial `tickers[]` array for unknown symbols rather than erroring — so ticker existence should be validated via the snapshot response itself (empty result for that symbol = invalid/unrecognized ticker), not via a separate call to `/v3/reference/tickers`.