audio: raise a turned-down microphone, and let the user fix what the model mishears - #22
Open
andredezzy wants to merge 9 commits into
Open
audio: raise a turned-down microphone, and let the user fix what the model mishears#22andredezzy wants to merge 9 commits into
andredezzy wants to merge 9 commits into
Conversation
andredezzy
changed the title
andredezzy
force-pushed
the
feat/raise-input-gain
branch
2 times, most recently
from
August 1, 2026 12:46
012a2b7 to
c2a60a2
Compare
UNVERIFIED -- not reproduced locally, do not upstream as-is. A long-lived AVAudioEngine reports the input format it cached at construction, so after the default device changes installTap gets a format the bus no longer has and AVFAudio raises 'com.apple.coreaudio.avfaudio': "Failed to create tap due to format mismatch, <AVAudioFormat 1 ch, 16000 Hz, Float32>". That is an ObjC exception, so the daemon dies rather than throwing. Confirmed the format is stale (engine reported 48000 Hz while the device sat at 44100 Hz) but installTap tolerates a rate-only divergence, so the failing condition needs a device identity change to reproduce.
Records from the chosen microphone without touching the system default input, so other apps are unaffected. The preference is stored as the device UID, not the AudioDeviceID: CoreAudio reassigns the numeric id across reboots and reconnects, so a stored id can silently point at a different microphone. It resolves to an id per recording, and an absent device falls back to the system default until it reappears. AudioCapture binds the node with auAudioUnit.setDeviceID(_:) before reading the input format, since the node reports the format of whichever device it is bound to. The submenu is rebuilt in menuWillOpen so a newly plugged mic shows up without watching CoreAudio for device changes.
CoreAudio creates an aggregate device per audio client the moment an input node is touched, named CADefaultDeviceAggregate-<pid>-N. It is visible only to that client, reports input channels, and is not flagged hidden — so parrot was listing its own plumbing as a selectable microphone. The composition dictionary of those aggregates carries the private key, which is what separates them from an aggregate the user built in Audio MIDI Setup; the latter is not private and stays listed.
AVAudioEngine's input node binds to whatever input is current the moment the
node is first touched, and nothing reconfigures it afterwards. Measured against a
Bluetooth headset while the built-in mic stayed the system default:
auAudioUnit.setDeviceID(_:) noErr, 0 frames in 3s
AudioUnitSetProperty(CurrentDevice) noErr, 0 frames in 3s
setDeviceID + engine.reset() noErr, 0 frames in 3s
headset as system default, no binding 115200 frames, peak 0.0212
Every binding call reports success and then delivers silence, which is how the
menu's device picker could look correct and record nothing.
A standalone AUHAL unit honours kAudioOutputUnitProperty_CurrentDevice: same
headset, same conditions, 48000 frames in 3s. Verified end to end afterwards —
5.06 s captured at rms 0.028 through the headset with the system default left on
the built-in microphone.
Two things leave the code with AVAudioEngine. AUHAL converts to 16 kHz mono
itself, so AVAudioConverter is gone; and there is no installTap(onBus:format:),
so the uncatchable AVFAudio format-mismatch exception that could terminate the
daemon on a device change is no longer reachable. The unit is disposed on stop so
a Bluetooth headset does not keep its microphone link open.
AUHAL input does not resample. Asking a 48 kHz device to deliver 16 kHz renders nothing at all: the built-in microphone captured 0.00 s while a 16 kHz Bluetooth headset — whose native rate already matched the request — captured fine. The first version of this only ever ran against the headset, so the conversion path was never exercised. Read the device's own format from the input scope, ask AUHAL for float at that same rate, and put AVAudioConverter back in charge of reaching 16 kHz mono. AUHAL then does only what it can do: pick the device and hand over float. `AudioUnitRender` failures are now logged. Swallowing that status is what made a failed render look exactly like a microphone that heard nothing.
Left unwritten, kAudioOutputUnitProperty_CurrentDevice still reports the right device and AudioOutputUnitStart returns noErr, but the unit renders zero frames. Measured: 0 frames in 2 s unbound against 30400 bound to the same device id. The system default input is now resolved and written like any other choice, so the default preference is no longer the broken one.
A MacBook's built-in microphone was found at 13% input gain, and that one setting is the largest measured accuracy factor in this project. Same speaker, same phrase, same model: gain 0.133 peak 0.083 baseline "revisá-los por request" gain 0.700 peak 0.375 +11.6 dB at 1.5-3.5 kHz "pull requests" correct gain 0.900 peak 1.000 566 clipped samples one dictation came back empty Across a whole corpus the difference is 84.8% -> 94.7% word accuracy. The consonants that separate words live in that band; below the floor they are not in the recording to begin with, and no decoder setting recovers them. Recording raises the gain to the measured optimum and puts the user's value back on stop. Devices already above 50% are left alone, and a device that refuses the write is left alone. 0.9 is deliberately not the target: the extra energy up there is clipping distortion, and it transcribes worse.
andredezzy
force-pushed
the
feat/raise-input-gain
branch
from
August 1, 2026 21:39
c2a60a2 to
9589e0e
Compare
Recording raises a turned-down microphone by itself, but a device that refuses the write does so silently, and the user would be left with a quiet recording and no reason for it. The check only speaks up in that case: adjustable devices are handled without saying anything.
andredezzy
force-pushed
the
feat/raise-input-gain
branch
from
August 1, 2026 21:58
9589e0e to
dac8224
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Raise a turned-down microphone for the length of a recording, and put the user's setting back afterwards.
Includes #14 as a dependency: the gain is applied to the resolved capture device, which is what the picker chooses.
My MacBook Pro microphone sat at 0.133 input gain, and nobody sets that on purpose. Measured on the same sentence, same voice, at three levels: 0.133 gives 0.083 peak, 0 clipped samples; 1.000 gives 566 clipped samples and +14.0 dB. The 1.5 to 3.5 kHz band, where the consonants that separate words live, came in 11.6 dB weaker than at 1.000. Whisper reads a 0.70 gain during the recording, and if the level is below 0.50 it restores your value on release.
Doctor reports a microphone parrot cannot turn up, because some devices expose no settable volume scalar.