Free app that speaks aloud while you sign.
Teach it any sign — from any sign language (ASL, BSL, Swedish teckenspråk, …) or your own home signs — and the camera speaks your phrase out loud whenever the sign is performed. Built as a companion to Communicator, for people who sign but whose surroundings don't.
Use it here: https://ecksquad.github.io/signspeaker/
- Press ✚ Teach a new sign, type what the sign should say.
- Perform the sign in front of the camera after a 3-2-1 countdown (about 2½ seconds). Record 3 examples for reliable recognition.
- That's it — whenever the sign is performed, the phrase is spoken aloud.
Because you teach the signs, it works with any sign language and any personal signing style — it is not limited to one country's sign vocabulary.
The app ships with a built-in vocabulary of ~190 everyday Swedish signs (needs, feelings, people, food, activities, places, time, animals, colours, question words…) that were learned automatically from Svenskt teckenspråkslexikon, Stockholm University's Swedish Sign Language dictionary. They load automatically — no setup — and can be renamed, disabled one by one, or switched off entirely in ⚙ Settings. A sign you teach yourself always wins over a built-in one.
The pack (learned/learned.json) is rebuilt with tools/learn_from_web.py, which does the whole pipeline on its own: wordlist → dictionary search → video download → hand-landmark extraction with the same MediaPipe model the app uses → app-format samples (each sign also gets a mirrored twin so left-handed / mirror-imitated signing matches):
pip install mediapipe opencv-python requests
python tools/learn_from_web.py tools/words-pack.txt -o learned/learned.json --append
Wordlist lines are word or word = phrase the app should speak (tools/words-pack.txt is the built-in pack's list, tools/words-start.txt a small starter). You can also harvest a private wordlist to a JSON file and load it via ⚙ → Import — imported learned signs merge in without touching your own taught signs.
The learned sign data in learned/learned.json is derived from the lexicon's videos and is therefore licensed CC BY-NC-SA 4.0 (attribution: Svenskt teckenspråkslexikon, Stockholm University) — unlike the app code, which is MIT. Downloaded videos stay on your machine and must not be redistributed.
The 🔤 button switches to ASL fingerspelling mode: spell a word letter by letter (static manual alphabet A–Y; J and Z need motion and are not included yet) and the whole word is spoken aloud when you lower your hand. Recognition is rule-based on hand-landmark geometry — the look-alike fist letters (M, N, S, T, E) are genuinely hard to tell apart and may need careful signing.
- Speaks in 7 languages — English, Svenska, Українська, Español, Deutsch, Français, Polski — with a male or female voice of your choice.
- Own voice: record any phrase with a real voice (e.g. a family member's) and it plays instead of the computer voice.
- Handles both still handshapes and moving signs, one- or two-handed.
- Adjustable strictness and rest time; export/import of all taught signs.
- Completely private: all recognition (MediaPipe hand tracking + on-device matching) happens locally in the browser. The camera image never leaves the device. No install, no account, no server.
Single HTML file. MediaPipe Tasks Vision (HandLandmarker, 2 hands, GPU with CPU fallback), landmark sequences matched with banded dynamic time warping, Web Speech API for TTS, MediaRecorder + IndexedDB for own-voice recordings, localStorage for signs and settings.
MIT