Server-side huddle recording with per-speaker audio, so transcripts become channel events agents can read

[artile]

Summary

Record each huddle server-side, with audio bound to each speaker's pubkey, so it can be transcribed and the transcript lands in the channel log as signed events. Today a huddle's spoken content is the one modality that never becomes part of the event log — so agents watching a channel can see that a huddle happened, but never what was said.

Current state (as I read the code)

• The relay only forwards audio; it persists nothing. crates/buzz-relay/src/audio/* mixes/routes live streams. The only huddle events written to the log are lifecycle metadata — KIND_HUDDLE_STARTED/PARTICIPANT_JOINED/PARTICIPANT_LEFT/ENDED (buzz-core/src/kind.rs). Reaction bursts are explicitly ephemeral and never stored.
• The relay already knows who is speaking. The audio room keys every participant by (pubkey, peer_index) and maintains a peer_index → pubkey roster (crates/buzz-relay/src/audio/room.rs). Per-speaker attribution is already inherent server-side — no diarization guesswork needed.
• Transcription exists, but only client-side and ephemeral. It runs on-device in the desktop app (desktop/src-tauri/src/huddle/{stt,transcription,pipeline}.rs, embedded sherpa-onnx + Parakeet English-only). It is not recorded, not durable, and not written to the log as events. (Huddle transcription is English-only: non-English speech (e.g. Korean) produces garbled transcripts #2478 tracks the English-only limitation; this request is orthogonal — it is about where the recording/transcript live, not which STT model runs.)

So the gap: no durable per-speaker recording, and no transcript in the event log.

Why it matters

Buzz's whole premise is "one event log, one search index — humans, agents, workflows all speak it." Huddles break that promise: their content evaporates. The "incident memory" story in the README ("an agent pulls six months of history") simply cannot include anything anyone ever said out loud. Making the transcript a first-class signed event closes the last modality gap, and it does so for agents specifically — an agent that is a channel member would get huddle content as context the same way it gets messages today.

Proposal

1. Server-side recording, per speaker. Persist each huddle's audio keyed by speaker pubkey (the relay already has (pubkey, peer_index)), to the existing media store (Blossom/S3). Per-speaker streams make diarization free and let transcription attribute every segment to an npub.
2. Transcription against the per-speaker audio. Run STT server-side (or via a pluggable/off-box transcriber) over each speaker's track. Because segments are already attributed, output is "speaker → text → time range" with no speaker-separation heuristics.
3. Transcript as signed events in the channel log. Emit transcript segments as signed events scoped to the huddle + channel, tagged with speaker pubkey and a timestamp range — searchable via the existing FTS index, and readable by agents as ordinary channel context. A dedicated kind (or an extension of the huddle-lifecycle range) keeps them identifiable without decrypting.

This composes cleanly with #2451 (evidence-backed knowledge crystallization as channel-scoped signed events): a huddle transcript is exactly the kind of evidence that RFC wants to crystallize.

Notes / considerations

• Consent is not optional. Recording a conversation is sensitive; it should be opt-in per huddle and visibly announced to participants. The existing KIND_HUDDLE_GUIDELINES (48106) is a natural place to carry a "this huddle is recorded" policy.
• Storage/retention. Media store already exists (S3/MinIO/Blossom); recordings need a retention policy and a way to redact/delete (a NIP-09-style deletion for the transcript events + object cleanup).
• Pluggable STT. Given Huddle transcription is English-only: non-English speech (e.g. Korean) produces garbled transcripts #2478, the transcriber should be swappable and language-aware rather than pinned. A per-speaker pipeline also lets different speakers use different language models where locale is known.

Happy to help — we run a Parakeet-based transcription pipeline elsewhere and would gladly test a per-speaker, server-side path (including non-English, which ties back to #2478).

[artile]

Opened #2522 toward this. It lands the protocol foundation: a KIND_HUDDLE_TRANSCRIPT (48104) signed channel event whose content is plain transcript text (FTS-indexed + agent-readable like a message), with speaker p / channel h / huddle e / start/end tags — plus a typed reader (buzz_core::huddle), an SDK builder, and relay ingest registration. Built + tested (9 tests).

Kept intentionally to the reviewable slice. The remaining pieces from your proposal — server-side per-speaker recording into buzz-media, transcription over the attributed tracks, and consent/retention via KIND_HUDDLE_GUIDELINES + NIP-09 deletion — are designed in the PR body as follow-ups so each PR stays small. Since attribution is already inherent server-side ((pubkey, peer_index) in the audio room), the recording step maps cleanly onto this event.