Skip to content

Add Grok STT as the default transcription provider - #130

Open
mattstyles333 wants to merge 2 commits into
browser-use:mainfrom
mattstyles333:grok-stt-default
Open

Add Grok STT as the default transcription provider#130
mattstyles333 wants to merge 2 commits into
browser-use:mainfrom
mattstyles333:grok-stt-default

Conversation

@mattstyles333

@mattstyles333 mattstyles333 commented Aug 12, 2026

Copy link
Copy Markdown

Why

video-use currently hard-depends on ElevenLabs Scribe. Grok STT already has the three things the cut pipeline needs: word-level timestamps, speaker diarization, and filler-word retention.

Vision does not need a new API. timeline_view.py still emits local filmstrip/waveform PNGs; the host agent (Grok, Claude, Codex, …) reads them with whatever vision it already has.

What

  • helpers/transcribe.py accepts either XAI_API_KEY or ELEVENLABS_API_KEY.
    • ElevenLabs-only .env (today's install) keeps working with no flags.
    • xAI-only uses Grok STT.
    • If both are set, Grok is used unless you pass --provider elevenlabs.
  • Grok words are normalized into the existing Scribe-shaped schema (type / speaker_id / synthesized spacing tokens) so pack/render/timeline_view stay unchanged.
  • install.md upserts one key line and never truncates .env.

Not in this PR

  • Optional Manim TTS (still ElevenLabs / Qwen if someone uses that skill)
  • No change to ffmpeg/PIL vision composites

Grok STT already ships word-level timestamps, diarization, and filler-word
retention. Normalize its words into the existing Scribe-shaped schema so
pack/render/timeline_view stay unchanged. ElevenLabs remains available
via --provider elevenlabs.

Vision composites stay local (timeline_view PNGs) — no API to swap.
Existing ELEVENLABS_API_KEY-only .env files keep working with no flags.
Either key is enough; if both are set, Grok STT is used unless
--provider elevenlabs. Install upserts one variable and never truncates .env.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 5 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="install.md">

<violation number="1" location="install.md:124">
P2: The new upsert path runs the user-pasted key through sed as the literal replacement text: `sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY}|"`. In the sed replacement, `&` expands to the entire matched text and `\` is an escape, and the chosen delimiter `|` collides if the key contains one. A key containing any of those characters (e.g. an ElevenLabs/xAI key with `&` or `|`) is silently written corrupted to `.env`, and transcription then fails against the bad key. Escape the key for the replacement context (e.g. `sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY//&/\\&}|"`) or write the line with awk instead so arbitrary key bytes are preserved.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread install.md
# KEY_NAME is XAI_API_KEY or ELEVENLABS_API_KEY
if grep -q "^${KEY_NAME}=" "$ENV"; then
tmp=$(mktemp)
sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY}|" "$ENV" > "$tmp" && mv "$tmp" "$ENV"

@cubic-dev-ai cubic-dev-ai Bot Aug 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The new upsert path runs the user-pasted key through sed as the literal replacement text: sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY}|". In the sed replacement, & expands to the entire matched text and \ is an escape, and the chosen delimiter | collides if the key contains one. A key containing any of those characters (e.g. an ElevenLabs/xAI key with & or |) is silently written corrupted to .env, and transcription then fails against the bad key. Escape the key for the replacement context (e.g. sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY//&/\\&}|") or write the line with awk instead so arbitrary key bytes are preserved.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At install.md, line 124:

<comment>The new upsert path runs the user-pasted key through sed as the literal replacement text: `sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY}|"`. In the sed replacement, `&` expands to the entire matched text and `\` is an escape, and the chosen delimiter `|` collides if the key contains one. A key containing any of those characters (e.g. an ElevenLabs/xAI key with `&` or `|`) is silently written corrupted to `.env`, and transcription then fails against the bad key. Escape the key for the replacement context (e.g. `sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY//&/\\&}|"`) or write the line with awk instead so arbitrary key bytes are preserved.</comment>

<file context>
@@ -96,40 +96,52 @@ Figure out which agent you are running under, and register once. A symlink of th
+    # KEY_NAME is XAI_API_KEY or ELEVENLABS_API_KEY
+    if grep -q "^${KEY_NAME}=" "$ENV"; then
+      tmp=$(mktemp)
+      sed "s|^${KEY_NAME}=.*|${KEY_NAME}=${KEY}|" "$ENV" > "$tmp" && mv "$tmp" "$ENV"
+    else
+      printf '%s=%s\n' "$KEY_NAME" "$KEY" >> "$ENV"
</file context>
Fix with cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found and verified against the latest diff

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="helpers/transcribe_batch.py">

<violation number="1" location="helpers/transcribe_batch.py:50">
P2: When a transcript already exists, `--provider` is ignored because the batch cache is provider-agnostic. Running `--provider grok` after an ElevenLabs transcript silently reuses the ElevenLabs result; track the provider in the cache decision or invalidate the cached file when it differs from the requested backend.</violation>
</file>

<file name="helpers/transcribe.py">

<violation number="1" location="helpers/transcribe.py:79">
P2: When a caller supplies an ElevenLabs `api_key` but omits `provider`, this ambient-key lookup can select Grok and send the wrong credential, causing authentication failures. Preserve the existing direct-call behavior or require and validate the provider alongside the key.</violation>

<violation number="2" location="helpers/transcribe.py:79">
P2: Adding a second provider makes the existing filename-only transcript cache return stale cross-provider data. A user who previously transcribed with ElevenLabs and now runs the new grok default will get their old Scribe transcripts back (and vice versa) because the cache checks only that `<video_stem>.json` exists. Make the cache provider-aware: include the provider in the cached filename or verify the stored `provider` field on the cache hit, and skip/re-transcribe when it differs.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

)
ap.add_argument("--workers", type=int, default=4, help="Parallel workers (default: 4)")
ap.add_argument(
"--provider",

@cubic-dev-ai cubic-dev-ai Bot Aug 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a transcript already exists, --provider is ignored because the batch cache is provider-agnostic. Running --provider grok after an ElevenLabs transcript silently reuses the ElevenLabs result; track the provider in the cache decision or invalidate the cached file when it differs from the requested backend.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At helpers/transcribe_batch.py, line 50:

<comment>When a transcript already exists, `--provider` is ignored because the batch cache is provider-agnostic. Running `--provider grok` after an ElevenLabs transcript silently reuses the ElevenLabs result; track the provider in the cache decision or invalidate the cached file when it differs from the requested backend.</comment>

<file context>
@@ -44,6 +46,12 @@ def main() -> None:
     )
     ap.add_argument("--workers", type=int, default=4, help="Parallel workers (default: 4)")
+    ap.add_argument(
+        "--provider",
+        choices=PROVIDERS,
+        default=None,
</file context>
Fix with cubic

Comment thread helpers/transcribe.py


def load_api_key(provider: str | None = None) -> str:
provider = resolve_provider(provider)

@cubic-dev-ai cubic-dev-ai Bot Aug 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a caller supplies an ElevenLabs api_key but omits provider, this ambient-key lookup can select Grok and send the wrong credential, causing authentication failures. Preserve the existing direct-call behavior or require and validate the provider alongside the key.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At helpers/transcribe.py, line 75:

<comment>When a caller supplies an ElevenLabs `api_key` but omits `provider`, this ambient-key lookup can select Grok and send the wrong credential, causing authentication failures. Preserve the existing direct-call behavior or require and validate the provider alongside the key.</comment>

<file context>
@@ -27,22 +30,53 @@
+
+
+def load_api_key(provider: str | None = None) -> str:
+    provider = resolve_provider(provider)
+    name = KEY_FOR_PROVIDER[provider]
+    v = _read_key(name)
</file context>
Fix with cubic

Comment thread helpers/transcribe.py


def load_api_key(provider: str | None = None) -> str:
provider = resolve_provider(provider)

@cubic-dev-ai cubic-dev-ai Bot Aug 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Adding a second provider makes the existing filename-only transcript cache return stale cross-provider data. A user who previously transcribed with ElevenLabs and now runs the new grok default will get their old Scribe transcripts back (and vice versa) because the cache checks only that <video_stem>.json exists. Make the cache provider-aware: include the provider in the cached filename or verify the stored provider field on the cache hit, and skip/re-transcribe when it differs.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At helpers/transcribe.py, line 75:

<comment>Adding a second provider makes the existing filename-only transcript cache return stale cross-provider data. A user who previously transcribed with ElevenLabs and now runs the new grok default will get their old Scribe transcripts back (and vice versa) because the cache checks only that `<video_stem>.json` exists. Make the cache provider-aware: include the provider in the cached filename or verify the stored `provider` field on the cache hit, and skip/re-transcribe when it differs.</comment>

<file context>
@@ -27,22 +30,53 @@
+
+
+def load_api_key(provider: str | None = None) -> str:
+    provider = resolve_provider(provider)
+    name = KEY_FOR_PROVIDER[provider]
+    v = _read_key(name)
</file context>
Fix with cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant