Skip to content

H-6763: Improve voice preview reliability diagnostics - #9378

Open
kostandinang wants to merge 3 commits into
kostandin/h-6763-openai-canonical-speechfrom
kostandin/h-6763-voice-preview-reliability
Open

H-6763: Improve voice preview reliability diagnostics#9378
kostandinang wants to merge 3 commits into
kostandin/h-6763-openai-canonical-speechfrom
kostandin/h-6763-voice-preview-reliability

Conversation

@kostandinang

@kostandinang kostandinang commented Aug 27, 2026

Copy link
Copy Markdown

🌟 What is the purpose of this PR?

This PR hardens the disabled-by-default OpenAI voice preview introduced by the lower H-6763 stack. It adds actionable, privacy-safe diagnostics and closes failure-handling gaps so developers can distinguish permission, device, network, timeout, abort, provider-response, transcription, and playback failures.

The scope is reliability and observability only. It does not enable production voice or add the authentication, distributed quotas, replay/recovery, completion, and projection contracts required for production use.

🔗 Related links

🚫 Blocked by

🔍 What does this change?

  • Defines a sanitized voice-error taxonomy covering microphone permission/device failures, aborted requests, network errors, timeouts, invalid responses, and unavailable preview endpoints.
  • Adds validated UUID request correlation plus content-free timing diagnostics across browser connection/transcription/playback, website voice routes, and the existing Brunch inspection join.
  • Adds valid Server-Timing metrics while explicitly excluding audio, SDP, transcripts, prompts, canonical speech, credentials, and provider response bodies from diagnostics.
  • Makes startup and teardown deterministic across timeout, disconnect, stale-event, malformed-response, provider-error, and reconnect paths; aborts remain classified as request-aborted while SDP is read or applied.
  • Prevents late answer delivery from overwriting active playback or a speech failure.
  • Adds a fully local mocked boundary test spanning WebRTC setup, transcript delivery, the composer/Brunch boundary, canonical response selection, speech streaming, and playback.
  • Updates the website README, Petrinaut user guide, ADR-0009, and Brunch steering documentation.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • require changes to docs which are made as part of this PR

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • Production voice remains disabled; this PR does not close the production gates recorded in ADR-0009.
  • Verification uses local mocks and fixtures. No live microphone, OpenAI Realtime/Speech request, or configured remote Brunch conversation was exercised.
  • Existing production-build chunk-size warnings are unchanged.

🐾 Next steps

  • Land after H-6763: Speak finalized Brunch responses with OpenAI #9357 as part of the H-6763 stack.
  • Address authentication, distributed quotas, replay/recovery, production telemetry, completion, and projection contracts before enabling production voice.
  • Capture a credentialed browser witness separately from deterministic automated coverage.

🛡 What tests cover this?

  • yarn workspace @apps/petrinaut-website test:unit — 17 files and 109 tests, including the mocked browser-to-playback boundary.
  • yarn workspace @apps/petrinaut-website lint:tsc and lint:eslint — website type and lint coverage.
  • Focused regressions cover startup, permission/device failures, network failures, request aborts, timeouts, malformed/provider responses, request-ID validation, teardown, reconnect/stale-event rejection, media cleanup, and late delivery races.
  • Brunch app and transport unit suites cover the inspection join used by voice diagnostics.
  • CI covers package builds, formatting, documentation, architecture checks, and Semgrep.

❓ How to test this?

  1. Check out this branch with H-6763: Speak finalized Brunch responses with OpenAI #9357 beneath it.
  2. Run yarn workspace @apps/petrinaut-website test:unit.
  3. Run yarn workspace @apps/petrinaut-website lint:tsc and yarn workspace @apps/petrinaut-website lint:eslint.
  4. Confirm the local integration test crosses the mocked browser → voice route → transcript → composer/Brunch → canonical speech → playback path.
  5. Confirm serialized diagnostics contain request IDs, timing, stage, operation, and sanitized error codes, but none of the private fixture payloads.
  6. Do not point tests at a remote Brunch endpoint or real OpenAI credentials without separate approval.

📹 Demo

Not included. This PR changes reliability, diagnostics, and automated coverage rather than the interaction design, and no credentialed external voice flow was run.

@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
petrinaut Ready Ready Preview Aug 27, 2026 5:01pm
3 Skipped Deployments
Project Deployment Actions Updated (UTC)
hash Ignored Ignored Preview Aug 27, 2026 5:01pm
hashdotdesign-tokens Ignored Ignored Preview Aug 27, 2026 5:01pm
petrinaut-docs Skipped Skipped Aug 27, 2026 5:01pm

Request Review

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team area/apps labels Aug 27, 2026
@CLAassistant

CLAassistant commented Aug 27, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ kostandinang
❌ ampagent
You have signed the CLA already but the status is still pending? Let us recheck it.

Comment thread apps/petrinaut-website/src/voice-diagnostics.ts
Comment thread apps/petrinaut-website/src/voice-diagnostics.ts Outdated
Comment thread apps/petrinaut-website/src/voice-diagnostics.ts Fixed
Comment thread apps/petrinaut-website/src/voice-diagnostics.ts Fixed

kostandinang commented Aug 27, 2026

Copy link
Copy Markdown
Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Classify actionable voice failures without exposing provider or conversation content, correlate browser and server requests, and record privacy-safe timing diagnostics. Add focused failure and cleanup coverage plus a fully mocked voice-to-Brunch-to-speech boundary test while keeping production disabled.

Co-authored-by: Kostandin Angjellari <ka@hash.ai>
Amp-Thread-ID: https://ampcode.com/threads/T-01a04317-a36c-7229-b43c-ad122f989238
@kostandinang kostandinang self-assigned this Aug 27, 2026
Co-authored-by: Cursor <cursoragent@cursor.com>
@cursor

cursor Bot commented Aug 27, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches real-time voice connection, speech streaming, and Brunch request headers; changes are preview-scoped with strong privacy guards but broaden failure-handling paths across browser and server.

Overview
Adds privacy-safe reliability diagnostics for the disabled-by-default Brunch voice preview: a shared voice-diagnostics layer with typed VoiceError codes, validated UUID x-request-id propagation, and content-free timing events (browser, server, playback) logged without audio, SDP, transcripts, or canonical speech text.

User-facing behavior replaces generic reconnect messages with actionable categories (microphone permission/device, network, timeout, abort, invalid response, unavailable). The status panel shows error code and diagnostic reference where appropriate; Brunch chat transport now sends x-request-id so FE-1505 can correlate the same boundary.

Server voice routes attach x-request-id, x-petrinaut-voice-error on failures, and Server-Timing metrics; failures no longer leak provider bodies. Turn controller carries errorCode / errorRequestId through recoverable states and fixes a race where late Brunch delivery could overwrite active playback or a speech failure.

Docs (README, Petrinaut guide, ADR-0009, steering) describe the taxonomy and clarify this is preview evidence only—not production PR 4. Coverage expands with unit tests plus a local mocked integration test across WebRTC → voice APIs → transcript → speech → playback.

Reviewed by Cursor Bugbot for commit 5bf7eda. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d1976e9. Configure here.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

4 participants