feat(meeting): real-time diarized 1:1 meeting capture bridge (phone -> Speechmatics) - #34
Merged
Merged
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2aaa7bdef3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| active_max_speakers=cfg.max_speakers, | ||
| active_speaker_id_enabled=False, | ||
| ) | ||
| return ActiveSession(ambient_cfg, source=source, speaker_id=None) |
There was a problem hiding this comment.
When the meeting bridge has two WebSocket sessions start in the same second (for example two phones, or a quick reconnect while the old connection is still finalizing), this factory gives both sessions the same cfg.output_dir and reuses ActiveSession unchanged. ambient_bridge/active_session.py names transcripts only as YYYYMMDDTHHMMSS.md, so the two sessions will share one path and their live _flush() calls can overwrite each other's transcript contents. Add a per-session unique component or serialize meeting sessions before reusing ActiveSession.
Useful? React with 👍 / 👎.
| [ | ||
| web.get("/capture/{token}", self._handle_capture), | ||
| web.get("/meeting/{token}", self._handle_ws), | ||
| web.get("/health", self._handle_health), |
There was a problem hiding this comment.
In the documented deployment the Tailscale Funnel exposes this same aiohttp app, so adding /health without the path token makes it reachable from the public URL even though /capture and /meeting are token-gated. The JSON includes active_sessions, frame/byte counters, last_frame_ts, and pid, which lets anyone poll the endpoint to learn when meetings are happening. Gate this route as well or serve health on a separate local-only listener.
Useful? React with 👍 / 👎.
…> Speechmatics) New standalone edge bridge `bridges/meeting_bridge/`: a phone streams 16k PCM over an authenticated WebSocket; the bridge relays it to Speechmatics for real-time streaming transcription + live diarization and writes a live-updating diarized `.md` — the capture precondition for Genesis acting on a meeting mid-call. Reuses the ambient bridge's proven, deployed `ActiveSession` (Speechmatics streaming + diarization + live `.md`) via a dependency-injected, env-pluggable session factory, so there's zero new cloud integration and the server unit-tests without the cloud SDK. Runs on its own port, deliberately separate from the 24/7 ambient service so a phone meeting-mic and the always-on home Voice PE never collide over ambient's bridge-level active/passive flag. - Single aiohttp app: GET /capture/<token> serves the phone PWA (getUserMedia -> AudioWorklet 16k downsample -> same-origin wss), GET /meeting/<token> is the audio WS. Constant-time path-token auth (+_PREVIOUS rotation); access_log=None keeps the token out of logs. - capture.html: streaming linear resampler in the worklet (iOS forces 48k), Screen Wake Lock, marker button; honest foreground-only. - Deploy: lean `meeting` install target (aiohttp+speechmatics-rt+numpy), systemd unit, meeting.env.example. Reuses the ambient bridge's existing Speechmatics key. - Tests (TDD): 11 passing — config/auth, WS PCM relay + marker + finalize, oversize guard, health, and a real-subprocess E2E guarding the token-never-logged property + graceful SIGTERM. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-leak guard - WS heartbeat (MEETING_WS_HEARTBEAT_S, default 20s): aiohttp now pings the phone and force-closes on a missed pong, so a silently-vanished peer (screen lock / tab kill / wifi handoff) is finalized in seconds instead of leaking an open, billed Speechmatics session with a stuck active-count. New test drives a silent (autoping=False) client and asserts finalize. - Move the session-factory call INSIDE the try so a factory/start failure is contained (logged, not propagated to aiohttp's error logger) and the finally cleanup always runs; finalize guarded for session is None. New test asserts no active-count leak + server stays healthy. - capture.html: wrap post-getUserMedia setup (AudioContext / worklet / ws) in try/catch that calls stop() on failure, so a worklet-load/ws error releases the mic instead of leaving it open with no stop button. - install.sh: clarify `both` = s2s+ambient (the meeting bridge is opt-in via `meeting`). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
WingedGuardian
force-pushed
the
feat/meeting-bridge
branch
from
July 11, 2026 17:36
2aaa7bd to
5dccfa1
Compare
…+ contract/setup docs Native Kotlin app (clients/android-mic) that streams the phone mic at 16 kHz mono PCM16 over the meeting bridge's authenticated WebSocket, from a microphone-typed foreground service so capture survives screen-lock and backgrounding — the ceiling the browser capture page couldn't clear. - MicStreamService: AudioRecord (VOICE_RECOGNITION) -> OkHttp WebSocket, persistent notification with Stop + Mark, partial wake lock, auto-reconnect (drops rather than buffers audio across a gap to keep live diarization in sync). - MainActivity: endpoint/token entry (baked via BuildConfig, gitignored secrets), Start/Stop, live status, and a Samsung battery-optimization exemption prompt (the device is a Samsung — it sleeps the service mid-meeting without the whitelist). - Endpoint + token never committed: only placeholders ship; real values inject at build time from app/secrets.properties / -P props / env. - Docs: CONTRACTS.md §1c (phone->edge meeting WS ingress), SETUP.md meeting section, client README (build + sideload + Samsung steps); .gitignore Android artifacts. Committed Gradle 8.9 wrapper; AGP 8.5.2 / Kotlin 1.9.24 / compileSdk 34 / minSdk 26. APK builds green; on-device E2E (locked-screen capture) is the remaining acceptance test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…art, cancel, volatile Code review (BLOCKED) findings on the mic client, all in the reconnect/restart path: - BLOCKER: MainActivity inferred RECORD_AUDIO from the permission-grants map, which omits it when only POST_NOTIFICATIONS was requested — so a user who granted mic but denied notifications could never start capture. Now checks actual mic-permission state. - START_STICKY restart delivered a null intent with no persisted creds, so the "resume after OS kill" path just errored out. Persist endpoint+token (app-private, cleared on explicit stop) and restore them on the null-intent restart. - Broad catch swallowed CancellationException, clobbering the STOPPED status with a bogus ERROR when the user stopped during a reconnect delay. Rethrow cancellation. - captureJob is read from OkHttp callback threads (onSocketDown) — marked @volatile for cross-thread visibility (matches wsConnected / the ws AtomicReference). Rebuilds green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…atus fix Client-driven Speechmatics model selection + two reliability/UX asks from on-device use: - Model toggle: the app picks standard|enhanced per session via a ?model= query on the WS URL; the bridge validates it against a whitelist and falls back to its configured default (MEETING_MODEL) for anything else — never trusts the query raw. The model is fixed for a Speechmatics session, so switching = stop & start. (bridge: _cfg_for_request + 2 tests.) - 8-hour safety auto-stop: a single continuous capture stops itself after 8h so a forgotten stream can't quietly run up cost. Resets on each Start / sticky restart. - Live-status fix: the status label + notification were stuck at "0s · 0 KB" because the StateFlow only emitted on phase changes; now refresh ~1/s while streaming (bytes really were flowing — verified on the edge). A frozen 0 now genuinely means "never connected". - Docs: CONTRACTS.md §1c ?model= param; client README behaviour notes (toggle, auto-stop, live status). Model choice + creds persisted for sticky-restart resume. Bridge tests 12 pass; APK rebuilds green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
versionCode 1->2, versionName 0.2.0, and a small version label in the app (v<name> (<code>)) so a sideload update is verifiable at a glance — a same- versionCode reinstall over a browser-cached APK silently no-ops, which reads as "nothing changed". The label is ground truth for which build is installed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The persistent notification is the primary status display while the screen is locked — the app's whole reason to exist — so a 10s-stale timer read as frozen. Bump the in-loop notification refresh from ~10s to ~2s. In-app label already ticks ~1s. (Source-only; no APK redelivery this round.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A new edge bridge
bridges/meeting_bridge/that captures a near-field 1:1 meeting in real time. A phone (Step 1, zero hardware) streams raw 16 kHz PCM over an authenticated WebSocket; the bridge relays it to Speechmatics for streaming transcription + live diarization and writes a live-updating, diarized.md— the capture precondition for Genesis acting on a meeting mid-call ("in two places at once"). Far-field/noisy-office capture stays the home Voice PE's job (ambient_bridge).Why Speechmatics (verify-first)
The ASR choice was settled by evidence, not assumption: the
ambient_bridgealready has a deployed, E2E-tested cloud streaming + diarization path (ActiveSession), and real past transcripts on the edge show it cleanly separating multiple speakers with high-quality transcription. So rather than a new integration, this bridge reusesActiveSessionunchanged via a dependency-injected, env-pluggable session factory (MEETING_SESSION_FACTORY) — zero new cloud code, and the server unit-tests without the cloud SDK.Design
GET /capture/<token>serves the phone PWA;GET /meeting/<token>is the audio WS. Same-originwssmeans one Tailscale Funnel port and no CORS/mixed-content.+ _PREVIOUSrotation); the browser can't set WS headers so the token rides the URL;access_log=Nonekeeps it out of logs.getUserMedia→ an AudioWorklet resamples the mic to 16 kHz Int16 (iOS forces 48 kHz) → binary frames; Screen Wake Lock; marker button. Honest: foreground-only (Step 2 = a dedicated always-on device).Deploy (separate, user-gated)
deploy/install.sh meeting(lean~/meeting-venv: aiohttp + speechmatics-rt + numpy) +deploy/systemd/meeting-bridge.service; secrets in~/.meeting/meeting.env(placeholders indeploy/meeting.env.example); reuses the ambient bridge's existing Speechmatics key. Exposed via the Tailscale Funnel.Tests (TDD)
13 passing: config/auth, WS PCM relay + marker + finalize, oversize-frame guard, health JSON, heartbeat-driven finalize of a silent peer, factory-failure containment, and a real-subprocess E2E guarding the token-never-logged property + graceful SIGTERM. Local
ruff/compileall/shellcheck -S error/ private-data scan all clean. A code review pass was addressed in the second commit (WS heartbeat, factory containment, mic-leak guard, install docs).Capture only; the live consumer/attention engine that acts on the transcript mid-meeting is future work.
🤖 Generated with Claude Code