H-6763: Add the voice interview experience - #9380
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
|
|
github-actions
Bot
added
area/infra
8014db4 to
f7f2f10
Compare
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
PR SummaryMedium Risk Overview The Brunch local-storage demo wires Docs and changesets describe the new interview behavior; tests cover stage remounting, answer buffering, coverage selection, and controller/session edge cases. Reviewed by Cursor Bugbot for commit 190c5d7. Bugbot is set up for automated code reviews on this repo. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Co-authored-by: Kostandin Angjellari <ka@hash.ai> Amp-Thread-ID: https://ampcode.com/threads/T-01a0431a-f233-7239-971b-ed234c72dc6c
Co-authored-by: Cursor <cursoragent@cursor.com>
f7f2f10 to
190c5d7
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 4 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 190c5d7. Configure here.
| JSON.stringify({ type: "input_audio_buffer.commit" }), | ||
| ); | ||
| } catch { | ||
| this.#handleConnectionFailure("network", "transcription"); |
There was a problem hiding this comment.
Done speaking can kill the session
High Severity
Done speaking always sends input_audio_buffer.commit while provider-default server VAD is still committing automatically. A second commit on an empty buffer, or a commit with no speech, returns a realtime error that this session treats as a fatal connection failure, so a normal end-of-answer click can drop the interview into reconnect.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 190c5d7. Configure here.
| this.#audioContext = audioContext; | ||
| this.#analyser = analyser; | ||
| this.#meterSamples = new Uint8Array(analyser.fftSize); | ||
| } |
There was a problem hiding this comment.
Microphone meter stays silent
Medium Severity
The input meter creates an AudioContext only after getUserMedia resolves, with no resume(). That runs outside the user-gesture window, so the context stays suspended in Chromium and Safari. getByteTimeDomainData then returns silence and the stage reports Quiet even while capture is actually on.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 190c5d7. Configure here.
| )} | ||
| </div> | ||
| </> | ||
| )} |
There was a problem hiding this comment.
Corrections discarded before they can send
High Severity
Redo answer and Edit text appear whenever lastCommittedText is set, including during delivering and waiting. The controller ignores both unless phase is listening and no answer is in flight. Submitting a typed correction also clears the form first, so an early send silently drops the user's text.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 190c5d7. Configure here.
| this.#update({ | ||
| errorMessage: "", | ||
| currentQuestion: "", | ||
| microphoneLevel: 0, |
There was a problem hiding this comment.
Ended interview keeps prior answer
Medium Severity
end() resets phase and the current question but leaves lastCommittedText in place. Starting a new interview before the next brunch_ask arrives still shows Redo answer / Edit text for the previous session, and using them sends a correction of that old answer into the new conversation.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 190c5d7. Configure here.
3 participants
🌟 What is the purpose of this PR?
Turn the H-6763 composer microphone into a persistent, half-duplex interview experience while preserving canonical Brunch speech, keyboard fallback, and the reliability work in the parent stack.
This draft is stacked on #9378.
🔗 Related links
🚫 Blocked by
brunch_askbefore background sweep/settlement. The transport and voice controller consume an emitted ask immediately, but current upstream ordering can still delay emission itself; the UI deliberately does not fake answer readiness before the canonical ask exists.🔍 What does this change?
@apps/petrinaut-website.brunch_asktext before settlement completes, and measures finalized-answer → question-visible/spoken/answer-ready latency.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
🐾 Next steps
🛡 What tests cover this?
corepack yarn workspace @apps/petrinaut-website test:unit— 120 testscorepack yarn workspace @hashintel/petrinaut test:unit— 219 testscorepack yarn lint:formatcorepack yarn workspace @local/petrinaut-arch-docs lint:arch-docs❓ How to test this?
📹 Demo
Not attached: the available browser environment did not have credentials for a truthful end-to-end voice recording.