menu bar: switch models, and keep only the one in use on disk - #21
Open
andredezzy wants to merge 7 commits into
Open
menu bar: switch models, and keep only the one in use on disk#21andredezzy wants to merge 7 commits into
andredezzy wants to merge 7 commits into
Conversation
WhisperKit's downloadBase defaults to ~/Documents/huggingface. With iCloud Desktop & Documents sync on, the model weights are replicated and, once the disk fills, evicted to dataless placeholders (SF_DATALESS, st_blocks=0). CoreML mmap()s the weight blob during ANE compilation, and mmap of an evicted file blocks indefinitely: the daemon prints "loading <model>..." and never reaches "listening on fn hold". A partially materialised read instead fails as CoreML error 3, "Failed to read first word from AudioEncoder.mlmodelc/ coremldata.bin. It is not a valid .mlmodelc file." Observed on macOS 26.4.1 with whisper-large-v3-turbo (1.6 GB) on a 94%-full disk. Point downloadBase at Application Support, which is neither user-visible nor sync-managed. Existing installs keep their Documents copy; note it on load so the space is reclaimable rather than silently abandoned.
Same model as whisper-large-v3-turbo, quantised by argmax to 632 MB instead of 1620 MB. Measured on an 8-utterance mixed Portuguese/English corpus: identical word accuracy (89.4%, the same 7 errors), with only two cosmetic differences across the eight transcriptions. Per-utterance latency is marginally worse (0.61–0.93 s versus 0.56–0.76 s): quantisation buys memory and disk, not compute, since the Neural Engine still unpacks the weights. Worth offering to anyone short on disk, which on a machine whose models were being evicted is not hypothetical.
andredezzy
force-pushed
the
feat/switch-models-from-the-menu
branch
from
August 1, 2026 21:39
0d4b0f8 to
0907413
Compare
Adds Parakeet TDT v3 alongside WhisperKit. Measured on the same eight recordings: an order of magnitude faster (0.07-0.11 s against 0.6-2.5 s per utterance) and better on short utterances carrying English technical terms, worse on long sentences. Which side of that trade matters depends on how a person dictates, so it is a choice in the menu rather than a new default. The model is picked from a Model submenu and remembered like the input device. `--model` still wins for the launch that passes it. Switching loads the incoming model before dropping the current one, so a failed download leaves the user dictating with what they had, and the preference is written only after the load succeeds. Whatever is not in use is deleted: 792 MB reclaimed switching away from large-v3-turbo-compressed, plus 11 MB of tokenizers that live outside the weight directory. The same sweep runs at startup, so models stranded by a crash or an older version come back as free space. whisperKitID becomes engineID: the field now holds Parakeet ids too.
Parakeet throws invalidAudioData under 0.25 s where WhisperKit returns an empty string, so a tap on the hotkey logged a failure instead of doing nothing. Measured: 0.1 s throws, 0.3 s returns "". The threshold comes from the engine rather than a number of ours, so it follows whatever the library decides its minimum is.
Deleting everything but the active model made switching cost a download each way — six minutes to try the other engine and six more to come back, which is not a choice anyone makes twice. The model switched away from is now kept, so going back is instant. Measured: Parakeet -> Whisper Base kept both on disk (461 MB + 145 MB, freed 0), and switching back took 6 s with no download. Everything else still goes: a first run with nothing remembered freed 657 MB. The purge moved out of ActiveTranscriber, which knew nothing about history, and into the two places that do — startup and the menu callback.
andredezzy
force-pushed
the
feat/switch-models-from-the-menu
branch
from
August 1, 2026 21:45
0907413 to
d6be843
Compare
Picking a model left the menu unchanged until the load finished, which for a model that is not on disk yet means minutes. The row you clicked kept the old tick, the state line kept the old name, and the only honest reading was that the click had been ignored. The row now takes a dash the moment it is clicked, the other rows go quiet until the switch settles, and the state line carries the download as a percentage. Both engines report the download and neither reports the load, so the line drops the number and says loading for the last stretch rather than sitting at 100%. WhisperKit only reports progress on its static download, so the weights are fetched first and the pipeline is pointed at the folder afterwards.
andredezzy
force-pushed
the
feat/switch-models-from-the-menu
branch
from
August 1, 2026 22:01
d6be843 to
74cbde4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Switch models from the menu bar, keep only what is in use on disk, and say what is happening while the switch runs.
Includes #12 as a dependency: the purge needs a known model root, and it cannot be
Documents.What it adds:
The last commit is about the click, not the model. Picking a model left the menu unchanged until the load finished, which for a model that is not on disk means minutes: the row kept the old tick and the only honest reading was that the click had been ignored. The row now takes a dash on click, and the state line carries the download as a percentage. Neither engine reports the load, so the line drops the number and says loading for the last stretch rather than sitting at 100%.
Verified on an M2: switching to a model not on disk showed 5%, 11%, 21%, 66%, then loading, then the new model active.