transcribe: select the audio track, and refuse to upload a silent one - #134
Open
DarkStyleee wants to merge 3 commits into
Open
transcribe: select the audio track, and refuse to upload a silent one#134DarkStyleee wants to merge 3 commits into
DarkStyleee wants to merge 3 commits into
Conversation
extract_audio ran without -map, so ffmpeg applied its default stream selection and took a single audio track — the first one. A multi-track recording is the normal case for a screen capture: OBS writes the application on track 0 and the microphone on track 1. Transcribing such a file uploaded the application audio and dropped the narration without a word about it. --audio-track selects the stream, zero-based, and defaults to 0, so existing single-track runs are unchanged. A file with more than one track says so in verbose output, since the default is right for some of them and wrong for others. The extracted wav is also checked for level before it is sent. A peak under -60 dBFS means the track is silent, which in practice means the wrong track was picked, and Scribe charges the same for silence as for speech. The error names the track count and points at the flag.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 2 files
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
…e the tracks Review on browser-use#134 caught four things. The transcript cache is keyed by video stem alone, and transcribe_one returns on a cache hit before it looks at audio_track. Rerunning with --audio-track 1 after a wrong-track run therefore handed back the very transcript the flag was meant to replace. The track goes into the file name now, and track 0 keeps the old name so existing transcripts stay valid. peak_dbfs read the whole take with readframes(getnframes()) and copied it into an array, so a two-hour 16 kHz mono file cost 230 MB twice over, with batch mode running several at once. It scans in 64k-frame chunks instead; measured on a real capture the peak is identical to the whole-file version. The "try --audio-track" hint flipped between 0 and 1, so on a file with three tracks it could point at another silent one. It lists the tracks that exist: "The file has 3 audio tracks; try --audio-track 0 or 2." The flag's help text said ffmpeg would otherwise take track 0. It applies its default stream selection, which picks the track with the most channels.
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
Review on browser-use#134 again: the previous commit put the track into the cache key in transcribe.py and left transcribe_batch.py testing for {stem}.json. Batch mode with --audio-track 1 therefore counted a file with a track-0 transcript as cached and skipped it, which defeats the flag, and never recognised the {stem}.track1.json it had just written, so it re-uploaded and re-billed that file on every run. Both now call transcript_path(), so the two cannot drift apart again. Verified on a directory holding one video and a track-0 transcript: the default run reports "1 cached, 0 to transcribe", the same run with --audio-track 1 reports "0 cached, 1 to transcribe" and goes on to the silence guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
extract_audioran without-map, so ffmpeg applied its default stream selection and took a single audio track. A screen capture normally has two: OBS writes the application on track 0 and the microphone on track 1. Transcribing such a file uploaded the application audio and dropped the narration without saying so. The result is a transcript of the wrong thing, not an error.--audio-trackselects the stream, zero-based, and defaults to 0. Note this makes the choice deterministic rather than preserving it exactly: ffmpeg's default selection picks the audio stream with the most channels, which is the first track when both are stereo, but not in general. A file with more than one track now says so in verbose output.The extracted wav is also level-checked before it's sent. A peak under -60 dBFS means the track is silent, which in practice means the wrong track was picked, and Scribe bills silence the same as speech. The error names the track count and points at the flag:
Verified on a two-track OBS capture where the microphone track is digital silence: the guard fires and nothing is uploaded.
Summary by cubic
Selects the audio track to transcribe, rejects silent tracks, and keys the transcript cache by track. Previously
ffmpeg’s default often picked app/game audio instead of the mic, and a cache hit could hide a wrong-track rerun.--audio-track(zero-based, default 0) totranscribe.pyandtranscribe_batch.py; uses-map 0:a:<n>inffmpegfor deterministic selection. For OBS mic, pass--audio-track 1. Single‑track files are unchanged.--audio-track.<video>.jsonfor track 0 and<video>.trackN.jsonotherwise. Single and batch modes sharetranscript_path(), so batch cache checks match writes and avoid re-uploads.Written for commit a6fdea1. Summary will update on new commits.