Support Flux multilingual and generate the Aura voice enum - #1
Open
mathisarends wants to merge 3 commits into
Open
Support Flux multilingual and generate the Aura voice enum#1mathisarends wants to merge 3 commits into
mathisarends wants to merge 3 commits into
Conversation
Version 5 of the SDK has no Flux options beyond the turn thresholds, so the multilingual model cannot be reached from it at all. Version 7 also reshapes the socket surface the adapters sit on: - `deepgram.extensions.types.sockets` is gone; the socket clients now expose typed `send_close_stream()` and `send_flush()` instead of a generic `send_control(message)`, and `SpeakV1Text` moved to `deepgram.speak.v1.types`. - The Flux fatal error message is tagged `Error`, not `FatalError`. - `SpeakV1Warning` renamed `warn_msg`/`warn_code` to `description`/`code`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PS7H1tuQAMgeU8wdyj3WAq
Flux has grown past the turn thresholds vocalbin forwarded. `flux-general-multi` went generally available in April 2026 and transcribes ten languages on one connection, biased by `language_hints`; alongside it Flux picked up `numerals`, `profanity_filter`, `redact`, and the `linear32`, `opus`, and `ogg-opus` encodings. Language hints are validated against the supported set and rejected for `flux-general-en`, which is English-only, so a bad combination fails before the socket is opened rather than as a 400 from the server. Turn events now carry the `languages` the multilingual model heard, the `languages_hinted` it was given, the audio window they cover, and per-word timings. Mid-stream `Configure` messages are deliberately left out: retuning thresholds or hints without reconnecting needs a handle on the live connection, which the `stream()` generator does not hand out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PS7H1tuQAMgeU8wdyj3WAq
The hand-maintained enum listed 51 voices — English and Spanish only — and had drifted well behind Deepgram, which now ships 91 Aura 2 voices including German, French, Italian, Dutch, and Japanese. Reading them out of the SDK's model literal, the way the Cartesia voices are generated, keeps the list honest across SDK upgrades instead of relying on someone noticing a release note. `AuraVoice` names the enum after the model family it belongs to, matching `FluxModel` on the speech-to-text side; `TextToSpeechModel` stays as an alias so existing call sites keep working. Aura 1 voices are left out: they predate the `aura-2-<name>-<language>` naming and are superseded by it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PS7H1tuQAMgeU8wdyj3WAq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flux was already wired up for
flux-general-enandflux-general-multi, but only with the turn thresholds. Everything Deepgram added around the multilingual model — generally available since April 2026 — was out of reach, becausedeepgram-sdkwas pinned to>=5,<6and v5'slisten.v2.connect()does not know those parameters at all.Commits
Migrate the Deepgram adapters to deepgram-sdk 7 — the pin move is what unlocks the rest. v7 also reshapes the socket surface:
deepgram.extensions.types.socketsis gone,send_control(message)became typedsend_close_stream()/send_flush(), the Flux fatal error is taggedErrorinstead ofFatalError, andSpeakV1Warningrenamedwarn_msg/warn_codetodescription/code.Expose the newer Flux options on the streaming client —
language_hintsfor the ten languagesflux-general-multihandles on one connection, plusnumerals,profanity_filter,redact, and thelinear32,opus, andogg-opusencodings. Hints are validated against the supported set and rejected for the English-onlyflux-general-en, so a bad combination fails before the socket opens rather than as a server-side 400. Turn events now carrylanguages,languages_hinted, the audio window they cover, and per-word timings.Generate the Aura voice enum from the SDK — the hand-maintained list had 51 English and Spanish voices; Deepgram ships 91, including German, French, Italian, Dutch, and Japanese.
scripts/generate_deepgram_voices.pyreads them out of the SDK's model literal, the way the Cartesia voices are generated.API
FluxModelandAuraVoicename the two model families; the genericStreamingSpeechToTextModelandTextToSpeechModelstay as aliases, so nothing existing breaks.examples/deepgram/flux_multilingual.pyruns a German and an English line through it end to end.Deliberately left out
Mid-stream
Configuremessages, which retune thresholds and hints without reconnecting. That needs a handle on the live connection, and thestream()generator does not hand one out — a separate change to the port.Verification
ruff check,ruff format, and the full suite pass at each of the three commits, with the 100% coverage gate met throughout (256 tests). The adapters are covered by fakes rather than live calls, so the v7 socket surface was checked against the installed SDK by hand — signatures, message tags, and field names — before the fakes were updated to match.🤖 Generated with Claude Code
https://claude.ai/code/session_01PS7H1tuQAMgeU8wdyj3WAq