Tennis

Webcam-controlled 3D tennis in the browser. Stand in front of a camera, swing, and play an opponent that quietly tracks how good you are.

TypeScript · React 19 · Three.js · MediaPipe Pose · WebRTC · Vitest — 313 tests, no game engine, no asset files

---

Overview

Most webcam games try to map your body onto a character and fail, because a webcam sees a 1×2 m box and a tennis court is 24 m long. This one doesn't try. The avatar runs to the ball on its own; you decide when to swing and how. That is the whole game, and everything else follows from it.

The interesting constraint is latency. A pose pipeline costs roughly 80–120 ms end to end, which is fatal in table tennis — the rally is over before the pipeline has reported where your arm is. In tennis the ball is airborne for 700–900 ms, so the same delay is about 13% of the reaction budget. The sport was chosen to fit the technology, not the other way round.

Underneath is a pure, deterministic simulation with no DOM and no Three.js in it: real ball physics with drag and Magnus force, a forward-solving shot engine, full tennis scoring, and a bot that plays through exactly the same input interface a human does. Because the engine is deterministic and its RNG is seeded, a whole match replays from nothing but a seed and a list of swings.

Table of contents

• Overview
• Key features
• Architecture
• Tech stack
• Prerequisites
• Getting started
• Controls
• Project structure
• Testing
• Design decisions
• Status and known limitations
• Deployment
• License

Key features

Swing detection from a webcam. Wrist velocity through a gesture arc, not the arm as a physics collider. Swing up for topspin, down for slice, level for a flat drive, reach high for an overhead. Lean to aim; swing harder for more pace.

Speeds measured in torso-lengths, not pixels. Normalising by shoulder-to-hip distance is what stops every threshold breaking when you stand closer to or further from the camera.

Calibrate to your own body. Press T, hit Calibrate to me, swing five times, and the thresholds are set from your actual swing speeds. Saved to the browser, so it's a one-time setup. Shot power also adapts silently as you play, for anyone who never opens the panel.

Real ball physics. Fixed 240 Hz timestep with gravity, quadratic drag and Magnus force, so topspin dives into the court and slice floats and skids. The bounce couples spin to horizontal velocity through surface friction.

An opponent that meets you where you are. Difficulty slides continuously along five axes (reaction, timing jitter, coverage, aggression, precision) instead of three fixed presets, converging on a target win rate between points.

Footwork that matters without deciding the point. Stepping across to a wide ball makes the shot come off better; standing rooted makes it worse. It never decides whether you connect — only how well.

Match replay. Every match records its seed and your swings, then re-simulates from them. Not a recording of the ball's positions: the same simulation happening again. Verified bit-exact at four different playback frame rates.

Pose inference off the main thread. Frames go to a Web Worker as transferred ImageBitmaps. The GPU delegate boots in ~110 ms and steady-state inference runs 14–16 ms, leaving the main thread for physics and drawing. Measured on a real camera at a steady 30 fps with no dropped frames.

Three ways to play, one interface. Keyboard, touch and camera all produce the same SwingEvent. So does the bot. The engine cannot tell them apart.

Head-to-head over WebRTC (experimental — see limitations). No server, no accounts; two browsers swap a copy-paste code.

Match presentation. Break/set/match point called out, games and sets announced, a reason given for every point you lose, and an end-of-match summary with winners, unforced errors, aces, double faults, longest rally and how many swings were on time.

Architecture

The load-bearing idea is the boundary in the middle of that diagram. Every input adapter — camera, keyboard, touch, bot — collapses into one small value:

type SwingEvent = {
  t: number;            // ms, when the swing peaked
  arc: SwingArc;        // 'low-to-high' | 'high-to-low' | 'overhead' | 'flat'
  power: number;        // 0..1, calibrated by the input adapter
  lateralBias: number;  // -1..1 from torso lean
  side: 'forehand' | 'backhand';
  stance?: number;      // -1..1, only the camera can see this
};

Everything below that line is pure. The engine imports no DOM API and no Three.js, which buys four things: it is testable headlessly with synthetic swings, it was playable by keyboard long before any vision code existed, the bot is not a special case, and swapping the pose model touches one directory.

Latency compensation lives only in the pose adapter. The engine grades swing timestamps at face value, because the bot and the keyboard have no sensor lag to undo — applying a camera's compensation to them would bias every one of their swings early.

DESIGN.md covers the reasoning in more depth, including the coordinate-frame rule that caused a real bug (aiming right sent the ball left) and why multiplayer is an authoritative host rather than lockstep.

Tech stack

Layer	Choice	Why
Language	TypeScript 5.7, strict + noUncheckedIndexedAccess	the engine is maths; the types are the guard rail
UI	React 19	HUD, menus and overlays only — never the game loop
Rendering	Three.js 0.180	WebGL court, avatars, ball, shadows
Pose	MediaPipe Tasks Vision 0.10.20 (pose_landmarker_lite)	the lite model deliberately; every ms of inference is a ms of the latency budget
Build	Vite 7	dev server, code splitting, worker bundling
Tests	Vitest 3 + Testing Library	313 tests split into node and jsdom projects
Netcode	Native WebRTC data channels	no server to deploy or keep running
Audio	Web Audio API	every sound synthesised at runtime, zero asset files

Prerequisites

• Node.js 20.19+ or 22.12+ (required by Vite 7) and npm
• A Chromium-based browser or Firefox with WebGL2
• A webcam — optional. Keyboard and touch play need no camera at all
• For camera play outside localhost: HTTPS, because getUserMedia requires a secure context

Getting started

git clone https://github.com/lumixed/tennis.git
cd tennis
npm install
npm run dev

Open http://localhost:5450.

Then:

1. Pick a starting opponent — Rookie, Club or Pro. It adapts from there, so this is only where it begins.
2. Pick a match length — Quick is one set to four games and finishes in a few minutes. Full is best of three.
3. Pick Keyboard or Camera (touch is detected automatically on phones).
4. Press Play. Or Practice to rally with no scoreboard, or Watch bots to see the simulation play itself.

If you chose Camera: allow the webcam prompt, stand back far enough that your hips and shoulders are both in frame, then press T and hit Calibrate to me. Do this first — it beats any constant shipped in the source, which is necessarily tuned to one body at one distance.

Other commands

npm run build      # typecheck, then production build
npm run preview    # serve the production build (needed to exercise the pose worker)
npm run typecheck  # tsc --noEmit
npm test           # 313 tests, once
npm run test:watch # watch mode

Controls

Camera

Gesture	Shot
swing upwards	topspin
swing downwards	slice
swing level	flat drive
reach above your shoulders	overhead / smash
lean left or right	aim
swing faster	more power
step across to a wide ball	better shot

Keyboard

Key	Action
J / Space	topspin
K	flat drive
L	slice
I	overhead
A / D	aim
hold, then release	charge power
M	mute
T	tuning panel
Esc / P	pause

Touch

Tap for a flat drive, flick up for topspin, flick down for slice, tap near the top of the screen for an overhead, drag sideways to aim, hold before releasing to charge. The camera pulls back on tall screens so the whole court stays in frame.

Project structure

src/
  engine/   pure, deterministic simulation — no DOM, no THREE, fully tested
  vision/   MediaPipe → SwingEvent; detector and worker; the detector is pure
  scene/    Three.js rendering; reads engine state, owns no logic
  input/    keyboard and touch adapters
  game/     session, replay recording, hit-stop and slow motion, settings
  net/      head-to-head over WebRTC; protocol and sync are pure and tested
  audio/    synthesised sound, no asset files
  ui/       HUD, tuning overlay, match summary, lobby, error boundary
docs/       demo gif and architecture diagram

Testing

npm test

313 tests, split into two Vitest projects: engine runs in Node, ui runs in jsdom. Roughly what they cover:

• Physics against reality — a ball dropped from 2.54 m rebounds inside the ITF-specified band; energy never increases in flight; friction always drives spin towards the rolling condition
• Scoring — deuce, advantage, tiebreak serve rotation, and a property test driving 300 randomised matches that must all end in a legal scoreline
• Bot balance — measured statistically over hundreds of points, because "Pro beats Club" is not something you can assert from the constants
• Gesture detection against synthetic pose sequences, including that the same physical swing measures the same at 30 fps and 60 fps, and at any distance
• Replay determinism — bit-exact reproduction, including through a JSON round trip and at four different playback rates
• Mirroring — mirrorSnapshot is asserted to be its own inverse, the property that stops two networked players drifting into different worlds

Design decisions

Why the arm is not a collider. Treating the arm as a physics body that the ball tests against is the intuitive design and it fails in practice: landmark jitter produces phantom hits, latency produces ghost misses, and every whiff reads as the game's fault. A measurement failure must never render as a bad performance. Reading velocity through an arc degrades gracefully instead.

Why filters are time constants, not per-frame blends. A fixed per-frame EMA smooths twice as hard at 30 fps as at 60 — the identical physical swing measured a peak of 7.8 on one webcam and 8.9 on another. Shot power would have depended on the player's hardware.

Why shots are solved forwards. Magnus force makes the trajectory non-analytic, so the solver sweeps elevation angles against the same stepBall used in play and takes the flattest one that clears the net and carries far enough. A predicted shot and a played shot therefore cannot drift apart.

Why multiplayer is an authoritative host. The engine is deterministic, but advance takes a variable delta, so two peers at different frame rates would apply swings at different points in the physics accumulator and drift with nothing to detect it. An authoritative host cannot desync at all, and costs one round trip on the guest's swing — which a 700–900 ms ball flight absorbs.

Status and known limitations

Feature-complete and playable. Three things are worth stating plainly:

• Play a friend is experimental. The protocol and host/guest sync are covered by tests — mirroring, snapshot interpolation, a full match with the scores agreeing on both ends — but those run over an in-memory loopback transport. net/webrtc.ts has no test coverage and has never run between two machines, so signalling, NAT traversal, latency and disconnects are unproven. There is also no TURN server, so two players both behind symmetric NAT will not connect.
• The pose worker only runs in a built bundle. MediaPipe's FilesetResolver calls importScripts, which module workers forbid, so the worker must be a classic one — and Vite's dev server always instantiates ?worker imports as module workers. Under npm run dev inference falls back to the main thread, which is why npm run preview exists. The tuning panel's thread readout shows which is live.
• The camera has only been played on one machine. A full match has been played on a real webcam (30 fps pose, 15.6 ms inference on the GPU delegate, no dropped frames), but that is a single body, room and camera. Thresholds that suit one player may not suit another, which is what Calibrate to me is for.

Deployment

The build is fully static — npm run build emits dist/, deployable to any static host.

Two things to get right:

• Camera play needs HTTPS. getUserMedia only works in a secure context. Netlify, Vercel and GitHub Pages all provide it.
• On a GitHub Pages project site (username.github.io/tennis), set base: "/tennis/" in vite.config.ts or every asset will 404. User sites and other hosts need no change.

Initial load is ~215 kB gzipped. The pose pipeline (~45 kB) and its worker load only when someone actually picks camera mode.

License

MIT.