docs: capture this session's work in the memory bank (PRs #16-#21)

.gitignore

@@ -61,3 +61,6 @@ web/dist/
 # npm wrapper
 npm/like-duh/node_modules/
 
+
+# TypeScript incremental build info
+web/tsconfig.tsbuildinfo

memory-bank/activeContext.md

@@ -1,10 +1,54 @@
 # Active Context
 
-**Last Updated**: 2026-06-21
-**Current Phase**: Token usage tracking + npm wrapper — committed and pushed to main
+**Last Updated**: 2026-06-22
+**Current Phase**: Catalog refresh + tooling, incremental persistence, Cloudflare/GLM provider, unified report — all merged to main (PRs #16–#21)
 **Next Action**: Continue feature work or address open questions
 
-## Latest Work (2026-06-21)
+## Latest Work (2026-06-22) — PRs #16–#21
+
+### Model Catalog Refresh + Refresh Tool (PR #16 catalog, #19)
+- Added frontier models (Opus 4.8/4.7, GPT-5.5, GPT-5.4 mini, Gemini 3.5 Flash); dropped deprecated o3
+- Fixed GA context windows (Opus 4.6 + Sonnet 4.6 → 1M; Sonnet 4.5 stays 200k — its 1M is beta-gated)
+- **Temperature sets** (the recurring gotcha): `NO_TEMPERATURE_MODELS` (OpenAI) gained `gpt-5.5`;
+  new `ANTHROPIC_NO_TEMPERATURE_MODELS` (Opus 4.8/4.7) fixed a production 400. `AnthropicProvider`
+  omits `temperature` for those in `send`/`stream`. The OpenAI no-temp set is coupled to
+  `_REASONING_EFFORT_MODELS` (effort=high forces temperature=default).
+- `scripts/refresh_catalog.py` — propose-only drift detector: diffs catalog vs truefoundry feed +
+  live APIs, hashes a per-model projection (`scripts/catalog_snapshot.json`), discovers new models,
+  and empirically probes OpenAI + Anthropic temperature. **Feed is reliable for price, wrong for behavior.**
+
+### Incremental Persistence + REST Unification (PR #16 persist, #17)
+- New `src/duh/memory/persist.py` `IncrementalPersister`: create thread `active` up front →
+  `persist_round()` commits each finished round → `finalize()` marks `complete` + attaches
+  overview/followups/usage. Mid-run crash leaves a real partial thread instead of nothing.
+- `ConsensusContext.snapshot_round()` extracted from `_archive_round`
+- WS streams `thread_started` (real id) early for mid-run deep-linking
+- REST `/api/ask` now persists the **full** debate via the shared path (was lite/decision-only);
+  `_run_consensus` gained an additive `on_thread_created` callback — 8-tuple return unchanged
+- 3 duplicated persist paths consolidated into the one module
+
+### Cloudflare Workers AI + Zhipu GLM-5.2 (PR #18)
+- `cloudflare` provider via Workers AI's OpenAI-compatible endpoint; `@cf/zai-org/glm-5.2` in catalog
+  (262k ctx, $1.40/$4.40 per Mtok)
+- `OpenAIProvider` generalized with optional `provider_id` → serves any OpenAI-compatible host
+- `.env`: `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_WORKERS_AI_TOKEN` → loader builds the base URL
+- Live-validated (text + JSON mode); shows in `duh models`
+
+### Unified Consensus Report + Overview in History (PR #20, #21)
+- Copy/Export moved to top of report; dropdown downward + opaque + click-outside
+- `ThreadDetailResponse` now returns `overview` (history previously couldn't show the executive summary)
+- New shared `ConsensusReport` component renders **both** live (`ConsensusComplete`) and history
+  (`ThreadDetail`) identically; export per-view via an `exportSlot`
+- Markdown parity confirmed (same `<Markdown>`; `.duh-prose` has no own font-size)
+
+### End-of-session state
+- **1677 Python + 204 Vitest tests**, mypy clean (63 files), ruff clean, build clean
+- **Open follow-ups**: (1) self-healing temperature retry as a cross-provider safety net (this bug
+  hit twice: gpt-5.5, then Opus 4.8); (2) gitignore `web/tsconfig.tsbuildinfo` (tracked build artifact)
+
+---
+
+## Prior Work (2026-06-21)
 
 ### Token Usage Tracking (end-to-end)
 - `ProviderManager` accumulates `total_input_tokens` / `total_output_tokens` alongside `total_cost`, all reset in `reset_cost()`
@@ -119,9 +163,9 @@
 
 ## Current State
 
-- **Branch `main`** — all work committed and pushed
-- All previous features intact (v0.1-v0.6, PR #13-#15)
-- 1657 Python + 204 Vitest tests passing, build clean
+- **Branch `main`** — all work committed and pushed (through PR #21)
+- All previous features intact (v0.1-v0.6, PR #13-#21)
+- 1677 Python + 204 Vitest tests passing, mypy clean (63 files), build clean
 
 ## Open Questions (Still Unresolved)
 

memory-bank/decisions.md

@@ -1,6 +1,6 @@
 # Architectural Decisions
 
-**Last Updated**: 2026-03-08
+**Last Updated**: 2026-06-22
 
 ---
 
@@ -438,3 +438,57 @@
 - Sequential challengers (defeats the purpose of multi-model)
 **Consequences**: First challenger to respond appears immediately. More engaging real-time experience. WS test mocks now patch `_stream_challenges` instead of `handle_challenge`. Challenge order in UI reflects completion speed, not configuration order.
 **References**: `src/duh/api/routes/ws.py:253-347`, `tests/unit/test_api_ws.py`
+
+---
+
+## 2026-06-22: Incremental Per-Round Persistence Over Single Write at COMPLETE
+
+**Status**: Approved (implemented)
+**Context**: All three persistence paths (CLI, WebSocket, REST) built the full thread in memory and wrote once at COMPLETE. A crash mid-run lost the entire consensus. REST additionally had a "lite" path that saved only the decision.
+**Decision**: A single shared `IncrementalPersister` (`src/duh/memory/persist.py`): create the thread `active` up front, commit each finished round in its own transaction, finalize to `complete` at the end. All three entry points use it. `_run_consensus` exposes the created thread id via an additive `on_thread_created` callback (keeping the 8-tuple return stable).
+**Alternatives**:
+- Keep single-write (simpler, but mid-run crash = total loss)
+- Write-ahead log / event sourcing (more robust, much heavier for a SQLite app)
+- Bump `_run_consensus` to a 9-tuple to return the id (breaks ~7 callers + test mocks)
+**Consequences**: Mid-run crash leaves a real partial `active` thread. Three duplicated implementations collapsed into one. Final DB state identical to before, so state-asserting tests stayed green.
+**References**: `tasks/2026-06/260622_incremental-persistence.md`, `src/duh/memory/persist.py`
+
+---
+
+## 2026-06-22: Empirical Behavior Probing + Propose-Only Catalog Refresh Over Trusting Feeds
+
+**Status**: Approved (implemented)
+**Context**: The model catalog goes stale (new models, price changes, deprecations). A community feed (truefoundry/models) has the data but is unreliable on *behavioral* fields — it was wrong about GPT-5.5 rejecting temperature.
+**Decision**: Model IDs come from live provider APIs; pricing/context/status from the feed, cross-checked against known anchors; **temperature/behavior is probed against the live endpoint, never trusted**. `scripts/refresh_catalog.py` is a manual, propose-only tool — it diffs and reports, never auto-edits `catalog.py`. Temperature membership is encoded in `NO_TEMPERATURE_MODELS` (OpenAI) and `ANTHROPIC_NO_TEMPERATURE_MODELS`.
+**Alternatives**:
+- Trust the feed wholesale (would have shipped a gpt-5.5 temperature bug)
+- Runtime fetch of the catalog (adds a third-party dependency to billing-critical cost math + an availability surface)
+- Self-healing retry on the "temperature deprecated" 400 (more robust; deferred as a cross-provider safety net — see open follow-up)
+**Consequences**: No fabricated catalog data. The refresh tool caught two of my own context-window mistakes during development. The static-set approach must be maintained per release — the tool now probes both OpenAI and Anthropic so drift is visible. Note: this class of bug bit users twice (gpt-5.5, then Opus 4.8).
+**References**: `tasks/2026-06/260622_model-catalog-refresh.md`, `scripts/refresh_catalog.py`
+
+---
+
+## 2026-06-22: One OpenAI-Compatible Adapter for Any Host (provider_id)
+
+**Status**: Approved (implemented)
+**Context**: `OpenAIProvider`'s `provider_id` was hardcoded to `"openai"`, so it couldn't serve a second OpenAI-compatible host (e.g. Cloudflare Workers AI) without colliding. Adding GLM-5.2 needed a distinct provider.
+**Decision**: `OpenAIProvider` takes an optional `provider_id` and resolves its catalog/capabilities from it. `_setup_providers` registers any enabled provider that has a `base_url` + key as an OpenAI-compatible adapter under its config name. Cloudflare is configured purely from `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_WORKERS_AI_TOKEN`.
+**Alternatives**:
+- A bespoke `CloudflareProvider` class (duplicates the OpenAI adapter)
+- Repoint the existing `openai` provider's base_url (would replace real OpenAI — can't run both)
+**Consequences**: Any OpenAI-compatible host (Cloudflare, Groq, Together, OpenRouter, AI Gateway) works by config alone. GLM-5.2 runs alongside the existing panel.
+**References**: `tasks/2026-06/260622_cloudflare-glm-provider.md`, `src/duh/providers/openai.py`
+
+---
+
+## 2026-06-22: One Shared ConsensusReport for Live and History Views
+
+**Status**: Approved (implemented)
+**Context**: The live view and the stored-thread view rendered the decision differently, and history couldn't show the executive summary (the thread-detail API never returned the overview).
+**Decision**: `ThreadDetailResponse` returns `overview`; a shared `ConsensusReport` component renders the decision block (meters, Copy/Export at top, executive-overview-first with the full decision in a disclosure, dissent) for both `ConsensusComplete` and `ThreadDetail`. Export differs per-view via an `exportSlot` (store-based dropdown live; shared `ExportMenu` for stored threads).
+**Alternatives**:
+- Keep two displays (the divergence the user objected to)
+- Force the live view to build a `ThreadDetail` object to reuse `ExportMenu` (awkward — no turns/contributions mid-run)
+**Consequences**: Identical rendering across both contexts; history finally shows the executive summary. Export stays per-view but in the same position.
+**References**: `tasks/2026-06/260622_unified-consensus-report.md`, `web/src/components/consensus/ConsensusReport.tsx`

memory-bank/progress.md

@@ -1,10 +1,24 @@
 # Progress
 
-**Last Updated**: 2026-03-20
+**Last Updated**: 2026-06-22
 
 ---
 
-## Current State: Thread View Parity + PDF Overhaul + Server Fixes
+## Current State (2026-06-22): Catalog, Persistence, Cloudflare/GLM, Unified Report
+
+Merged to `main` via PRs #16–#21 (details in `tasks/2026-06/README.md` and `activeContext.md`):
+- **Model catalog refresh** + propose-only `scripts/refresh_catalog.py` drift tool; OpenAI + Anthropic
+  temperature correctness (`NO_TEMPERATURE_MODELS`, `ANTHROPIC_NO_TEMPERATURE_MODELS`)
+- **Incremental persistence** (`IncrementalPersister`) across CLI/WS/REST — mid-run crash leaves a real partial thread
+- **Cloudflare Workers AI provider** with Zhipu GLM-5.2; `OpenAIProvider` generalized to any OpenAI-compatible host
+- **Unified `ConsensusReport`** for live + history; history now shows the executive summary
+- **Prior (2026-06-21)**: token usage tracking end-to-end + `npm/like-duh` wrapper
+- **Tests**: 1677 Python + 204 Vitest, mypy clean (63 files), ruff clean, build clean
+- **Open follow-ups**: self-healing temperature retry (cross-provider); gitignore `web/tsconfig.tsbuildinfo`
+
+---
+
+## Prior State: Thread View Parity + PDF Overhaul + Server Fixes
 
 ### Thread View Parity + PDF Overhaul + Server/UI Fixes (2026-03-20)
 

memory-bank/tasks/2026-06/260622_cloudflare-glm-provider.md

@@ -0,0 +1,51 @@
+# 260622_cloudflare-glm-provider
+
+## Objective
+Add Zhipu **GLM-5.2** as a usable model via Cloudflare Workers AI, and in doing
+so generalize the OpenAI adapter to serve any OpenAI-compatible host. (PR #18.)
+
+## Outcome
+- **`cloudflare` provider** backed by Cloudflare Workers AI's OpenAI-compatible
+  endpoint, with `@cf/zai-org/glm-5.2` in the catalog (262,144 context,
+  $1.40 / $4.40 per Mtok, per the Workers AI model page).
+- **Generalized `OpenAIProvider`**: gained an optional `provider_id` (default
+  `"openai"`) and resolves its model catalog + capabilities from that id (was
+  hardcoded). So it can run as `cloudflare` *alongside* real OpenAI without a
+  provider_id collision. This now enables ANY OpenAI-compatible host (Groq,
+  Together, OpenRouter, AI Gateway) by config alone.
+- **`_setup_providers`** (app.py): generic branch — any enabled provider with a
+  `base_url` + key registers as an OpenAI-compatible adapter under its config name.
+- **`.env`-driven**: `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_WORKERS_AI_TOKEN`.
+  The loader builds the account's Workers AI base URL
+  (`https://api.cloudflare.com/client/v4/accounts/<id>/ai/v1`) and configures the
+  provider. Missing either → no provider (graceful).
+
+## Files
+- `src/duh/providers/openai.py` — instance-based provider_id / catalog / caps
+- `src/duh/providers/catalog.py` — `cloudflare` capabilities + GLM-5.2 entry
+- `src/duh/cli/app.py` — generic base_url+key registration branch
+- `src/duh/config/loader.py` — `_resolve_cloudflare()` from env vars
+- `.env.example` — documents `CLOUDFLARE_*`
+- Tests: `test_providers_openai.py` (custom provider_id), `test_config.py`
+  (cloudflare env wiring), `test_cli.py` (registration)
+
+## Validation (live, against the user's Cloudflare account)
+- Provider wires up from `.env`; `@cf/zai-org/glm-5.2` appears in `duh models`
+- Plain text generation works (`finish_reason=stop`, token accounting correct)
+- **JSON mode works** — important because COMMIT and follow-up generation use
+  `response_format`
+- Note: GLM-5.2 is a reasoning model that spends tokens thinking, so it needs
+  real `max_tokens` headroom (duh uses 32k budgets — fine).
+
+## Usage
+```
+duh ask "..." --proposer cloudflare:@cf/zai-org/glm-5.2
+duh ask "..." --challengers cloudflare:@cf/zai-org/glm-5.2,openai:gpt-5.5
+```
+
+## Patterns
+- One adapter, many hosts: `OpenAIProvider(provider_id=name, base_url=...)`.
+- Test-isolation gotcha: `load_config()` runs `load_dotenv()`, which finds the
+  repo `.env` via the caller frame (not cwd). Tests asserting a provider is
+  *absent* must set the env vars to `""` (load_dotenv won't override existing,
+  even empty, vars), not `delenv` them.

memory-bank/tasks/2026-06/260622_incremental-persistence.md

@@ -0,0 +1,49 @@
+# 260622_incremental-persistence
+
+## Objective
+Replace the fragile "write everything once at COMPLETE" persistence with
+durable, incremental per-round persistence, and unify all three entry points
+(CLI, WebSocket, REST) onto one shared path. (PR #16 persist portion, PR #17.)
+
+## Problem
+Every path built the full thread in memory and did a single write at the end.
+A crash mid-run lost the entire consensus. Three separate implementations
+existed (`ws._persist_consensus`, `cli.persist_consensus`, `ask._persist_result`
+— the last a "lite" path that saved only the decision).
+
+## Outcome
+- **New `src/duh/memory/persist.py`** with `IncrementalPersister`:
+  - `start(question) → thread_id` — create the thread as `status="active"` up front
+  - `persist_round(RoundResult)` — write that round's turn + contributions
+    (+ citations) + decision, each in its own committed transaction
+  - `finalize(overview, followups, usage)` — flip to `complete`, attach summary/
+    followups/usage
+  - plus a `persist_consensus(...)` convenience that does start → rounds → finalize
+- A crash in round 2 of 3 now leaves a real `active` thread with 2 rounds, not nothing.
+- `ConsensusContext.snapshot_round()` (extracted from `_archive_round`) lets the
+  loop persist a finished round before it's archived to `round_history`.
+- **WebSocket** streams a `thread_started` event with the real thread ID up front,
+  so the client can deep-link mid-run.
+- **REST `/api/ask` unified (PR #17)**: now persists the *full* debate via the
+  same `IncrementalPersister` (was the lite decision-only path). `_run_consensus`
+  gained an additive `on_thread_created` callback so REST surfaces the thread ID
+  without changing the 8-tuple return (the other 7 callers untouched). The dead
+  `_persist_result` was removed.
+- The old `ws._persist_consensus` and `cli.persist_consensus` now delegate to the
+  one shared module (DRY — net negative lines in the affected files).
+
+## Files
+- Created: `src/duh/memory/persist.py`, `tests/unit/test_persist.py`
+- Modified: `src/duh/consensus/machine.py` (`snapshot_round`), `src/duh/cli/app.py`,
+  `src/duh/api/routes/ws.py`, `src/duh/api/routes/ask.py`, `tests/unit/test_api_ask.py`
+
+## Patterns
+- Persister methods each open their own session and commit independently → a
+  persisted round survives a later crash.
+- Final DB state is identical to the old single-write path, just written
+  progressively — so state-asserting tests stayed green.
+- Scope note: REST `/api/ask` is now on the full incremental path; CLI and WS too.
+
+## Outcome Metrics
+1663 → 1675 Python tests across the work (6 new persistence tests), mypy + ruff
+clean. Live-validated.

memory-bank/tasks/2026-06/260622_model-catalog-refresh.md

@@ -0,0 +1,59 @@
+# 260622_model-catalog-refresh
+
+## Objective
+Refresh the model catalog to the current frontier, correct per-model
+temperature behavior, and build a manual tool so the catalog stays accurate
+without fabricating data. (PR #16 catalog portion, PR #19.)
+
+## Outcome
+- **Frontier models added**: Claude Opus 4.8/4.7, GPT-5.5, GPT-5.4 mini,
+  Gemini 3.5 Flash. Dropped deprecated **o3** (per the feed's `status` field).
+- **Context windows corrected**: Opus 4.6 + Sonnet 4.6 → 1M (GA); Sonnet 4.5
+  stays 200k (its 1M is beta-gated — the live API reports the beta max, the
+  feed reports the GA default).
+- **OpenAI temperature**: added `gpt-5.5` to `NO_TEMPERATURE_MODELS` +
+  `_REASONING_EFFORT_MODELS`. Confirmed gpt-5.2/gpt-5.4 are *correctly* no-temp
+  because `reasoning_effort: high` forces temperature=default (the two sets are
+  coupled — a reasoning-effort model must also be no-temperature).
+- **Anthropic temperature (PR #19, production bug fix)**: Opus 4.8/4.7 deprecated
+  `temperature` and returned a 400. Added `ANTHROPIC_NO_TEMPERATURE_MODELS`;
+  `AnthropicProvider.send`/`stream` omit temperature for those. Older models
+  (Opus 4.6, Sonnet, Haiku) still accept it.
+- **Manual refresh tool** `scripts/refresh_catalog.py` (propose-only, never
+  auto-applies): diffs catalog vs the truefoundry/models feed + live provider
+  APIs, hashes a per-model field projection against `catalog_snapshot.json` to
+  show changes since last run, discovers new frontier models, and **empirically
+  probes** OpenAI *and* Anthropic temperature support against the live endpoints.
+
+## Files Modified
+- `src/duh/providers/catalog.py` — frontier models, `NO_TEMPERATURE_MODELS`,
+  new `ANTHROPIC_NO_TEMPERATURE_MODELS`, dropped o3
+- `src/duh/providers/openai.py` — `gpt-5.5` in `_REASONING_EFFORT_MODELS`
+- `src/duh/providers/anthropic.py` — omit `temperature` for no-temp models in
+  `send()` and `stream()`
+- `tests/unit/test_providers_google.py`, `test_providers_openai.py`,
+  `test_providers_anthropic.py` — catalog count + temperature-handling tests
+
+## Files Created
+- `scripts/refresh_catalog.py` — manual catalog refresh / drift detector
+- `scripts/catalog_snapshot.json` — per-model projection hashes (baseline)
+
+## Key Lessons (why this matters)
+- **Empirical > feed for behavior.** The truefoundry feed was *right* on pricing
+  (every anchor matched) but *wrong* on gpt-5.5 temperature. Behavioral fields
+  (temperature, reasoning_effort) must be verified against the live endpoint;
+  data fields (price/context/status) are diffed and human-confirmed.
+- **The Anthropic bug was a process miss**: when Opus 4.8/4.7 were added, only
+  OpenAI temperature was probed — the refresh tool now probes Anthropic too so
+  it can't recur silently.
+- GA-vs-beta context: live `/v1/models` reports the beta-gated max; the feed
+  reports the GA default. Use the GA default for the catalog.
+
+## Sourcing Rule
+IDs from live provider APIs; pricing/context/status from the truefoundry feed,
+cross-checked against known anchors; temperature/behavior probed live. Nothing
+fabricated.
+
+## Outcome Metrics
+1675 Python tests, 204 Vitest, mypy clean (63 files), ruff clean. Anthropic
+fix verified live (Opus 4.8 send/stream succeed).