An AI story engine that generates interactive narratives and rewrites them in real time based on user intent.
Not a single game — a platform. Drop in a story graph (JSON), and the engine handles intent classification, world state, branching logic, and cinematic asset generation. What the player says and does actually changes the story.
Built for the YC Google Hackathon.
Most "interactive stories" are glorified choose-your-own-adventures: fixed branches, scripted choices. The narrative doesn't respond — it just follows a flowchart.
Spielberg is an engine that:
- Classifies user intent — Voice or text → Gemini NLU → structured intent (attack, flirt, help, deceive, etc.) + tone + emotion + suggested narrative direction
- Maintains world state — Relationships (trust, tension, attraction), macro futures (6 story trajectories), threat level, emotional temperature, inventory
- Resolves transitions — Intent + world state → scored scene transitions. The story branches probabilistically based on what you've done, not just what you said last
- Generates assets on demand — Scene images (Gemini), character portraits, cutscenes (Veo), adaptive music (Lyria), character voices (ElevenLabs)
Stories are data. A story graph is a JSON file: scenes, transitions, characters, valid intents. Swap the graph → completely different narrative. Same engine.
┌─────────────────────────────────────────────────────────────────┐
│ React Frontend (Vite) │
│ CinematicPlayer • HUD • Voice input • Asset playback │
└────────────────────────────┬────────────────────────────────────┘
│ WebSocket
┌────────────────────────────▼────────────────────────────────────┐
│ Node.js / Express Server │
│ │
│ story-engine.ts → Story graph, transitions, world state │
│ intent-classifier → Gemini NLU → ClassifiedIntent │
│ live-api.ts → Gemini Live (narrator voice) │
│ asset-pipeline.ts → Image/video gen queue, precompute branches │
│ lyria.ts → Adaptive music streaming │
└────────────────────────────┬────────────────────────────────────┘
│
┌────────────────────┼────────────────────┬─────────────────┐
▼ ▼ ▼ ▼
Gemini Live Gemini 2.5 Lyria RT ElevenLabs
(narrator) (images) (music) (char voices)
Stories live in data/story-graphs/. Each graph defines:
- Scenes — Location, dramatic goal, characters, valid intents, transitions
- Transitions — Target scene, required intents, macro future affinity, condition flags
- Characters — Personality, voice ID, appearance
- Macro futures — 6 narrative trajectories (e.g.
pivot,viral,lawsuit,honest,failure) that shift in weight as the player acts
Included stories:
| Story | Genre | Hook |
|---|---|---|
| Series A-ccident | Dark tech comedy | Founder hit by a Waymo. 3 min before ambulance. Silicon Valley fever dream. |
| Noir Detective | Neo-noir | Rain-soaked PI office. A woman at the door. "They're going to kill me tonight." |
Add a new story = add a new JSON file + world bible. The engine runs it.
User says: "I'm gonna make this go viral"
Engine classifies:
- Primary intent:
advance(pushing the scene forward) - Tone: calculating
- Emotion: determined
- Suggested macro future:
viral - Narrative response: "Every crisis is a pivot. Every pivot is content. The blood on my forehead is already trending."
World state updates: macroFutureWeights.viral ↑, relationships shift. Next scene transition scores viral-affinity branches higher. The story bends toward that ending.
| Layer | Tech |
|---|---|
| Backend | Node.js, Express, WebSockets, TypeScript |
| Frontend | React, Vite |
| AI | Gemini Live API, Gemini 2.5 (images + NLU), Lyria RealTime, ElevenLabs TTS |
| Video | Veo 3 (cutscenes) |
cp .env.example .env
# Add GOOGLE_AI_API_KEY (or GEMINI_API_KEY) and ELEVENLABS_API_KEY
npm install
npm run dev- Server: http://localhost:3001
- Web: http://localhost:5173
Optional: Pre-generate scene assets for faster playback:
npm run pregenerate # Images + video
npm run pregenerate:images # Images onlyWe're not building a story. We're building the engine that makes stories respond. User intent drives narrative change. Different stories, same pipeline. That's the hack.
Sean, Swati, Alex, Shivam — YC Google Hackathon