Live Translate

English | 简体中文 | 日本語 | Español | Français

For real-time speech translation. A host speaks into the microphone; audience members join on their phones or laptops and hear translated audio plus live transcripts in their chosen language.

The app uses Gemini Live Translate through a local Node server. Your API key never reaches the browser.

How it works

flowchart LR
  Host["Host browser"]
  Server["Local server"]
  Gemini["Gemini Live Translate<br/>(one stream per target language)"]
  Audience["Audience browsers"]

  Host -->|"PCM audio"| Server
  Server --> Gemini
  Gemini --> Server
  Server -->|"translated audio + transcripts"| Audience

Loading

• Host creates a session, picks target languages, shares a room link or QR code, then starts speaking.
• Audience opens /rooms/:roomId, picks a language, and listens.
• The server opens one Gemini stream per target language, and only while at least one listener is connected for that language.

The host flow starts before anyone speaks: choose one or more target languages, open the room, and share the QR code or room link with listeners.

Language setup	Ready to speak	Share room

Audience members join the same room, pick their listening language, then see source and translated transcript lines paired by utterance while they wait for or hear the host.

Pick language	Bilingual transcript	Waiting for host

Prerequisites

• Node.js (via Vite+ / vp)
• pnpm
• A Gemini API key

Setup

pnpm install
cp .env.example .env

Add your key to .env:

GEMINI_API_KEY=your_key_here

Development

pnpm dev

This starts two processes:

Service	URL
Vite client	http://0.0.0.0:5174
API / WebSocket server	http://127.0.0.1:8787

Vite proxies /api and /ws to the server. Open the client URL on the host machine, then share the audience link from the host UI.

Run client and server separately if needed:

pnpm dev:server   # API only on :8787
pnpm dev:client   # Vite only on :5174

Production (local)

Build the SPA and serve everything from the Hono server:

pnpm build
pnpm start

Scripts

Command	Description
pnpm dev	Start API server and Vite dev client
pnpm build	Typecheck and build the client
pnpm start	Run the production server
pnpm test	Run tests
pnpm check	Format, lint, and typecheck

Project structure

src/
  client/          # Browser audio capture/playback and API helpers
  components/      # React UI (shadcn/ui)
  pages/           # Host and audience views
  server/          # Hono API, WebSocket rooms, Gemini integration
  shared/          # Shared types and protocol constants
docs/
  live-session-protocol.md
  live-translation-local-plan.md
  screenshots/       # Phone-sized showcase screenshots for the README

Protocol

HTTP and WebSocket message shapes are documented in docs/live-session-protocol.md.

Stack

• Vite+ — build, lint, format, test
• React + shadcn/ui
• Hono — HTTP API
• ws — WebSocket server