A four-seat AI writers' room that watches YouTube with you and reacts in real time. A fact-checker keeping the host honest. A sound-effects guy scoring every moment. A comedy writer dropping one-liners. A cynical troll saying what the audience is thinking.
Peanut Gallery is a Chrome Manifest-V3 extension. It captures the active tab's audio silently (chrome.tabCapture — no permission picker, no playback interference), streams PCM to a local or hosted backend, transcribes with Deepgram Nova-3, and routes each chunk through the Smart Director — a Claude-powered routing brain that picks which of four personas gets to fire next (with a rule-based safety net). Text reactions stream back via SSE and stack in the native Chrome side panel right next to the video.
Built in response to Jason Calacanis and Lon Harris's $5K open bounty on This Week in Startups. The TWiST pack puts Jason, Molly Wood, Lon Harris, and Alex Wilhelm on the panel — inspired by, not impersonating, with anti-impersonation guardrails baked into every prompt. Each persona ships with an illustrated peanut mascot (v1.5.3) with Phong-shaded lighting and animated reactions (v1.6.0).
git clone https://github.com/Sethmr/peanut.gallery.git
cd peanut.gallery
./setup.sh # installs deps + scaffolds .env.local for your API keys
npm run dev # local backend on http://localhost:3000Load the extension in Chrome:
- Open
chrome://extensions - Toggle Developer mode (top right)
- Click Load unpacked → select the
extension/folder at the repo root - Open any YouTube video → click the 🥜 icon → Start Listening
Your keys live in .env.local on your machine — git-ignored, never uploaded. Most users don't need to clone anything: install the latest build from the Chrome Web Store.
Prerequisites: Node.js 18+. No yt-dlp or ffmpeg required for the extension flow.
Four ways to use Peanut Gallery — pick whichever fits in the side panel's Backend & keys drawer:
| Mode | Cost | What it does | Who it's for |
|---|---|---|---|
| Demo | Free (one-off 15 min) | Hosted backend, one lifetime trial | Just trying it out |
| Peanut Gallery Plus | $8/month · 16 h/week | Hosted backend with a license key | Non-technical regulars |
| My keys (BYOK) | Free forever | Your Deepgram + Anthropic + xAI keys → hosted backend | Developers with API accounts |
| Self-host | Free forever | Your keys + your own server (npm run dev or Railway) |
Full control / privacy |
Plus is an accessibility lever, not a profit center — anyone who wants can always BYOK for free. Full details + US-only terms on peanutgallery.live/pricing.
Every pack ships four archetype slots. The Director is pack-agnostic — same routing, same cascade, same cooldowns — only the voices change.
Inspired by the Stern staff, per Jason's original spec.
| Slot | Character | Model | Role |
|---|---|---|---|
| Producer | Baba Booey (Gary Dell'Abate) | Claude Haiku + xAI Live Search | Fact-checker. Pulls receipts mid-show on numbers, dates, attributions. |
| Troll | The Cynical Troll | xAI Grok 4.1 Fast | Contrarian. Internet-brain energy. Says what the audience is thinking. |
| Sound FX | Fred Norris | xAI Grok 4.1 Fast | Bracket-delimited sound cues plus deadpan one-liners. |
| Joker | Jackie Martling | Claude Haiku | Setup-punchline jokes, callbacks, observational comedy. |
Sample fires:
Baba Booey: [FACT CHECK] "Jason just said Uber was founded in 2007. It was 2009. Again."
Troll: "Oh cool, another AI wrapper. Very 2024."
Fred: [record scratch] Fun fact: that company went bankrupt in 2023. [sad trombone]
Jackie: "Jason's investment thesis: if it has 'AI' in the name and the founder has a pulse, it's a yes."
Researched from public TWiST transcripts and episode clips. See docs/packs/twist/RESEARCH.md for the characterization source.
| Slot | Character | Model | Role |
|---|---|---|---|
| Producer | Molly Wood | Claude Haiku + xAI Live Search | Calm journalistic corrections, "according to" framing, receipts-first. |
| Troll | Jason Calacanis | xAI Grok 4.1 Fast | Provocateur. Confident takes, founder-market-fit framing, warm-not-mean. |
| Sound FX | Lon Harris | xAI Grok 4.1 Fast | The Reframe. Bracket-delimited sound cues plus cultural analogies. |
| Joker | Alex Wilhelm | Claude Haiku | Data Comedian. Eight joke techniques built on data plus absurdity. |
Pack choice lives in the side-panel setup dropdown and persists across sessions. Change takes effect on the next Start Listening — no mid-session persona swap.
Want another pack? Open a pack request — or author one yourself per docs/PACK-AUTHORING-GUIDE.md (living doc: archetype slots, producer contract, factCheckMode dial, registration, refinement loop). A visual Pack Lab authoring surface is deferred to post-v2.0; until then the guide plus the two reference packs in lib/packs/ are the template.
YouTube Tab → chrome.tabCapture → Offscreen Doc → PCM 16kHz (250ms chunks)
→ Backend → Deepgram Nova-3 → Director → AI Personas → SSE → Side Panel
The Director is the moat. It reads each transcript chunk and picks the best persona to respond, then cascades to others with decreasing probability and staggered timing. Some moments get 1 reaction, some 2-3, occasionally all four pile on. As of v1.6.0 ("The Canary") the primary router is a Claude Haiku tool_use call with verbalized confidence, sticky-agent penalty, unstable-tail heuristic, a live-callback ring buffer, and SILENT as a first-class choice — all behind a 400 ms budget with the rule-based scorer as the safety net. A fast-model shadow (Cerebras Llama 3.1 8B; Groq also wired) logs its pick alongside so we can verify agreement + latency before swapping primaries. The canary is flag-gated: ENABLE_SMART_DIRECTOR_V2=true on the backend turns the LLM router on; off keeps everything rule-based.
The Fact-Checker has an extra step: it scores sentences for factual claims and runs parallel search queries via xAI Live Search (Grok Responses API with the web_search tool) to cross-reference. Sensitivity is per-pack design: Howard's Baba Booey is loose (fires on speculation + name-drops + predictions), TWiST's Molly Wood is strict (hard claims only). One xAI key covers both persona generation and fact-check grounding.
If no audio is detected for 60 s, the extension auto-stops the session so it doesn't keep burning backend tokens on a paused tab. Press Start Listening again to resume.
Full architecture, SSE event protocol, and cost table: docs/CONTEXT.md.
The extension speaks to any backend that honors the wire spec. Three paths:
Point the extension at https://api.peanutgallery.live and bring your own API keys via the side-panel settings. Your keys are sent per-request as headers, forwarded to Deepgram / Anthropic / xAI, and discarded at session end. Never logged, never persisted. Audit the route.
The Next.js backend in this repo ships with a Dockerfile (yt-dlp + ffmpeg pre-installed for the legacy URL-paste endpoint). Railway recommended:
npx @railway/cli login
npx @railway/cli init -n my-peanut-gallery
npx @railway/cli upOr any Docker host:
docker build -t peanut-gallery .
docker run -p 3000:3000 peanut-galleryNo server-side env vars needed — users provide their own keys through the UI. Full operator's guide: docs/SELF-HOST-INSTALL.md.
Don't want to run Node? Want to swap the persona stack, change providers, or ship a branded fork in Go / Rust / Python? The extension will happily talk to anything that honors the wire spec.
docs/BUILD-YOUR-OWN-BACKEND.md— the canonical contract. Endpoint shapes, SSE event protocol, audio format, director + cascade rules, persona prompts, required CORS headers, 8 acceptance tests you can paste into curl.
Or let Claude do it for you. Paste this prompt into Claude Code after cloning — it reads the spec, scaffolds the project, and stops at the acceptance tests so you can verify compatibility before going live.
Multi-provider by design. No platform trap.
| Layer | Tech | Why |
|---|---|---|
| Backend | Next.js 15 App Router + Edge runtime | SSE streaming + free Vercel tier |
| Transcription | Deepgram Nova-3 | Sub-300ms, WebSocket native |
| Fact-Checker + Joker | Anthropic Claude Haiku | Reasoning + nuance |
| Troll + Sound FX | xAI Grok 4.1 Fast (non-reasoning) | Reflexive, punchy output without deliberation |
| Fact-check search | xAI Live Search (Grok Responses API + web_search tool) |
Reuses the xAI key — no separate provider |
| Smart Director (v1.6 canary) | Claude Haiku tool_use, 400 ms budget |
Primary LLM router; rule-based scorer is the safety net. Cerebras Llama 3.1 8B runs as read-only shadow for agreement + latency validation. |
Cost: ~$1 per hour at current API rates (Deepgram + Haiku + Grok Fast + one Grok Live Search call per fact-check candidate). Full breakdown: docs/CONTEXT.md#cost-per-episode.
All services have free tiers.
| Key | Sign up | Required? |
|---|---|---|
| Deepgram | console.deepgram.com | Yes |
| Anthropic | console.anthropic.com | Yes — powers the Fact-Checker + Joker |
| xAI | console.x.ai | Yes — powers the Troll + Sound FX AND the Producer's fact-check grounding via Grok Live Search (one key, two jobs) |
Canonical source: docs/ROADMAP.md. This table is what actually shipped; version numbers reflect reality, not earlier plans.
| Version | Theme | Date |
|---|---|---|
| v1.2.0 | Mise en place — Director debug panel + structured routing logs + fixture harness + pre-merge gate | 2026-04-17 |
| v1.3.0 | TWiST Pack — selectable packs, Howard + TWiST voices, pack swap in side panel | 2026-04-14 |
| v1.4.0 | Grok & Stability — xAI migration for troll + sound FX, search-engine toggle, session deadlock fix | 2026-04-17 |
| v1.5.0 | The Broadsheet — tabloid side-panel rebuild, mute-a-critic, night theme, Markdown export | 2026-04-19 |
| v1.5.1 | Broadsheet Final — 6-submenu settings drawer (absorbs old v1.6 "Settings Pane"), free-tier status strip, ON AIR, per-mug waveforms, round mugs | 2026-04-20 |
| v1.5.2 | First Run — four-step Editor's Note onboarding tour, empty-state visibility fix | 2026-04-20 |
| v1.5.3 | The Cast — illustrated peanut mascots for all 8 personas (absorbs old v1.8 "Peanut Mascots"), war-defense guardrail on fact-checkers | 2026-04-20 |
| v1.5.4 | The Sweep — SEO refresh, legacy /watch deletion, a11y pass, ops logging, orphan-dep cleanup |
2026-04-20 |
| v1.5.5 | Dev-infra — Linear ticket → Claude Code kickoff pipeline, framework + dev-dep refresh, release-model rewrite | 2026-04-20 |
| v1.5.6 | Dev-infra — local Linear daemon on Seth's Mac (replaces GH-Actions kickoff, uses Claude Max subscription) | 2026-04-20 |
| v1.6.0 | The Canary — Smart Director v3 flag-gated canary (Haiku tool_use, 5-slot with SILENT, Cerebras/Groq shadow), fact-check gate with per-pack sensitivity, fallback telemetry + self-correcting penalty loop, peanut avatar stage 1 (Phong lighting + unclipped bottoms), empty-state companions, director debug panel, 60s silence auto-stop |
2026-04-21 |
| v1.7.0 | The Fine Print — legal hard-save: rewritten ToS/Privacy drafts, US-only gate on Plus, cookie-consent banner, email aliases. Tagged + branched; held back from main pending lawyer review | 2026-04-21 |
| v1.8.0 | The Press Pass — Peanut Gallery Plus live end-to-end: SQLite identity (SET-25), Stripe checkout + webhook (SET-26), Resend email (SET-27), dedupe gate with one-click recover-key modal, four-mode access picker (Demo / Plus / My keys / Self-host), 15-min one-off free tier. Plus feed-UI polish (Regenerate, fire-count chips, session timer, transcript pulse) + fact-check hardening + privacy feedback opt-out | 2026-04-22 |
| v2.0.0 | The Gallery — CWS launch. v1.8.0 "Press Pass" Plus pipeline + v1.7.0 legal hard-save + v1.6.0 Smart Director canary all cleared to main. Persona v1.8 deep-research kernels for all 8 personas. Fact-check layer methodology (docs/FACT-CHECK-LAYER.md) — four-tier CONFIRMS / CONTRADICTS / COMPLICATES / THIN taxonomy applied to Baba + Molly via the new layered-fact-checker scaffolding, voice-agnostic. Inspired-by parody frame — persona.inspiredBy field on every named-person persona, with a prepended PARODY FRAME injected at fire time closing the anti-impersonation hedge promised on the marketing site. Audio polish (mute-SFX toggle in Audio drawer, Jackie↔Jason cue swap, bowling-pin cue for Jason, Fred + Lon cue trims, cue volume reduced to 0.67). Feedback & bugs drawer section (bug → GitHub, feedback → email). Director-v3 shadow log-noise reduction (Cerebras schema-dump demotion + stringified/partial-schema confidence coercion). |
2026-04-23 |
| Version | Theme | Status |
|---|---|---|
| v2.1.x | Baba fact-check tuning — live-log-driven pass-rate rebalancing for the new fact-check layer (first application landed in v2.0 but is more conservative on fire than live data wants). Potential Cerebras model swap (Llama 3.1 8B → 3.3 70B) for cleaner structured-output shadow telemetry. | Deferred post-ship per Seth |
| v2.2.x | Smart Director GA — once the v1.6 canary clears agreement + latency bands, promote the LLM router to primary and retire the rule-based scorer. Per-pack directorHint calibration from canary telemetry. Kill-switch flag stays. |
Blocked on canary data |
| v2.3.x | Persona refinement sprint 2 — re-run kernels against 100+ transcripts per pack after v2.0 launch feedback; tune anything the director says is under-firing. | Blocked on v2.1 tuning |
| v2.4.x | Avatar stage 2 — bobbleheads / 2.5D parallax / Lottie / MP4 fallback, whichever reads best in a 2-day spike. | Planned |
| v2.x.x | Continuous Director + persona improvements while we wait for user-driven v3 direction | Post-launch |
| v3.0.0 | User-driven — direction defined by what v2.0 users ask for, not by us | TBD |
Already supported, not yet marketed: non-YouTube sources (Twitch, Kick, any browser tab). chrome.tabCapture is tab-agnostic; personas are tuned for podcast pacing so it's "works, not pitched" until a pack tunes for a different format.
Read .github/CONTRIBUTING.md. The short version: ship specific, useful PRs. One logical change per PR. Fixtures for any director-routing change. Screenshots for any UI change. Risk tier on every PR.
- Bugs: report here — include version, backend, pack, repro steps, console.
- Features: request here — lead with the problem, not the solution.
- New persona packs: pack request — the four-archetype mapping forces the right shape early.
- Security: don't open a public issue. See
.github/SECURITY.md. - Help:
.github/SUPPORT.mdis the triage guide. - Community norms:
.github/CODE_OF_CONDUCT.md.
Peanut Gallery is built in the open on nights and weekends. If you get value from it, sponsor Seth on GitHub. Sponsors get early builds, pack-request priority, and a real thank-you in release notes.
Seth Rininger — iOS dev turned AI builder. 12+ years shipping apps at scale. Built for This Week in Startups — Jason Calacanis and Lon Harris.
Marketing site (separate repo, zero build step, iterates on a copy-and-design cadence): Sethmr/peanut.gallery.site → served at www.peanutgallery.live.
MIT. Fact-checking powered by xAI Live Search (Grok Responses API). Peanut Gallery is not affiliated with YouTube, Google, SiriusXM, or TWiST — personas are inspired by, not impressions of.