Development

Development

简体中文 · Home

Toolchain and build

The crate uses Rust edition 2024 and commits Cargo.lock. Project build and validation targets enforce the lockfile:

cargo build --release --locked
# Equivalent project target:
make build

Run a daemon from source only after stopping the installed service because both use the same runtime sockets:

systemctl --user stop voice-input.service
VOICE_INPUT_ASSET_DIR="$PWD/assets" cargo run --locked -- daemon

The source daemon still reads the normal user configuration and runtime directory. Restart the installed service when finished.

Repository layout

src/
  main.rs                   TLS setup and top-level exit handling
  args.rs                   hand-written CLI parsing and help
  app.rs                    command dispatch, diagnostics, Settings launch
  daemon.rs                 control socket, capture, session/finalization pipeline
  config.rs                 TOML schema, defaults, validation, migration
  diagnostics.rs            schema-4 privacy-safe support data
  credentials.rs            systemd/environment resolution and encryption
  focused_window.rs         bounded Hyprland/Kitty target discovery
  agent_context.rs          start-time Pi/Codex terminology snapshots
  backend.rs                ASR traits, controls, and events
  backend/
    local_cli.rs             /usr/bin/voxtype adapter and process boundary
    qwen_realtime.rs         Qwen Realtime WebSocket adapter
    qwen_batch.rs            compatible full-audio HTTP pass
    qwen_audio3/
      streaming.rs           experimental Streaming protocol and reconnect
      native.rs              experimental Native full-audio pass
    text.rs                  transcript extraction and OpenCC conversion
  http_client.rs            bounded sanitized HTTP helper
  llm.rs                    destination-aware refinement and shared deadline
  output.rs                 Wayland/XWayland clipboard delivery and helpers
  settings_backend.rs       versioned Settings NDJSON and persistence
  state.rs                  state machine and atomic private persistence
  waveform.rs               PCM analysis and Unix-socket publisher
  wav.rs                    bounded temporary PCM16 WAV handling
  paths.rs, setup.rs         installed paths and setup helpers
assets/
  config.toml                canonical public sample/default reference
  voice-input*.service      systemd user-unit templates
  quickshell/                resident HUD QML and shader source
  quickshell-settings/       on-demand Settings QML
  pi/                        Pi session-registry extension
  omarchy-*.conf/jsonc       Hyprland and Waybar snippets
docs/                        Audio3 decisions/evaluations and experiments
.github/workflows/ci.yml     CI validation

Keep assets/config.toml, Rust defaults and migration, Settings defaults/UI, both READMEs, and both Wiki languages synchronized when public behavior changes. The resident HUD and on-demand Settings are separate Quickshell configurations with different lifecycles.

Validation

Run all standard local checks:

make validate
make hud-shaders QSB=/usr/lib/qt6/bin/qsb
/usr/lib/qt6/bin/qsb --dump target/quickshell/shaders/wavy-halo.frag.qsb
git diff --check

make validate runs import-aware qmllint, Rust formatting, an all-target check, tests, and default Clippy lints with warnings denied. Set QMLLINT or QSB when Qt tools are installed elsewhere.

CI also parses every QML asset with Qt 6.8.3 qmlformat; this syntax check complements rather than replaces local qmllint. The shader job requires exactly six QSB targets—SPIR-V 100, GLSL 100 es, GLSL 120, GLSL 150, HLSL 50, and MSL 12—and verifies reflection for uniform block buf at binding 0, qt_Matrix at offset 0 as a 64-byte mat4, and qt_Opacity at offset 64 as a 4-byte float.

Automated coverage

Rust tests cover, among other boundaries:

• current/legacy TOML, Audio3 endpoint and final-pass migration, presets, vocabulary limits, and field validation;
• CLI parsing, bounded control commands/responses, connection admission, stale idle generations, and restart semantics;
• Qwen Realtime and Audio3 event parsing, transcript assembly, reconnect/replay policy, timestamp metadata, Native selection, and result precedence;
• schema-4 diagnostics allowlists, aggregate bounds, and sanitized provider identifiers;
• LLM response validation, destination prompts, terminology-only retry, OpenRouter sorting, and the shared 1–30 second deadline;
• Pi/Codex session validation, redaction, segmentation, low-frequency ordering, consumer limits, and local technical-term normalization;
• PCM16 fragmentation including odd-byte reads, waveform chunk independence, reusable output buffers, symmetry, NDJSON framing, and publisher backpressure;
• local-ASR and output child-process timeouts, process-group termination, concurrent stdout/stderr draining, output caps, executable discovery, clipboard-only routing, and sanitized errors;
• private/atomic state persistence and versioned Settings requests, exact-source revision conflicts, full-field validation, credential keep/replace, and restart-failure reporting.

Tests do not exercise a real microphone, compositor, remote provider account, Kitty/Pi/Codex process, Quickshell rendering, clipboard manager, or complete systemd credential lifecycle. QML is linted and parsed, but behavior such as malformed-frame recovery and multi-monitor rendering still needs manual integration checks.

Manual integration checklist

Use non-sensitive audio/text, a disposable clipboard value, and an account where remote test charges are acceptable.

1. Start both user services; verify control.sock, state.json, and waveform.sock metadata without printing state, then run voice-input diagnostics.
2. Test record start, stop, toggle, cancel, and restart; confirm restart is ignored while idle and starts a replacement only for an active session.
3. Test the shipped F8/F9/F10 Hyprland bindings and any hold-mode bind/bindr pair, including modifier release and controls queued during finalization.
4. Verify arming, live partial text, waveform visibility, the 350 ms no-speech grace, final empty handling, and automatic stop at audio.max_duration_secs.
5. Exercise Qwen Realtime normal completion, one controlled reconstruction, delivery overload, full-audio final pass, and local fallback independently.
6. For experimental Audio3, test Regional and Custom routing, region-scoped credentials, presets, language hints, heartbeat, vocabulary, one 4× replay reconnect, and Streaming-only/Adaptive/Always Native policies.
7. Isolate Audio3 APIs with voice-input asr stream-test --file … and voice-input asr test --file …; these commands send remote requests and may incur charges.
8. Verify native Wayland and XWayland paste targets, sensitive Wayland payloads with a compatible clipboard manager, restoration of a disposable clipboard item, helper timeout behavior, and Fcitx5 restoration.
9. Move/reset the HUD; test focus changes, multiple monitors, hotplug/reconnect, daemon-only restart, HUD-only restart, and malformed/stale state or waveform input without replacing the last valid UI state.
10. Restart/reload Pi after installing the extension. Start dictation in Pi/Codex, change focus before stop, and confirm start-time terminology and stop-time destination style remain independent.
11. Open Settings twice and confirm activation of one window. Test config-only save, credential keep/replace, Test LLM, stale revision conflict, partial restart failure, request timeout, backend crash, malformed/oversized/wrong-ID responses, bounded automatic restart, and manual Reload recovery.
12. Confirm voice-input diagnostics --format json contains no transcript, endpoint, model, credential, vocabulary term, Agent source, or provider message. Never attach status output or an unreviewed journal.

Design rules to preserve

• Fail open to recognized text: LLM failure must not discard valid ASR output.
• One refinement deadline: prompt/context work and any transcript-only retry share one budget capped at 30 seconds; budgets of at least 10 seconds reserve five seconds for recovery.
• No capture backpressure: capture must never wait for remote ASR or HUD clients. Audio-control and waveform queues are bounded and nonblocking; overload falls back to complete buffered audio. ASR event channels are currently standard unbounded Rust mpsc channels and must not be documented or assumed to be bounded without a separate redesign.
• Bound every external boundary: commands, responses, files, network messages, process output, recordings, retries, and waits need explicit limits plus timeout/disconnect/partial-data behavior.
• Sanitize errors: do not expose provider bodies, child stderr, transcript, clipboard content, credentials, terminology, or session data through user-facing errors or routine logs.
• Freeze operation context: start-time terminology is immutable and separate from stop-time destination classification. Preserve validation, redaction, consumer-specific caps, and prompt isolation.
• Secrets stay on stdin: Settings replacements travel QML stdin → Rust → systemd-creds stdin, never TOML/argv/environment/log/response. Clearing managed QML strings remains best effort.
• Rust owns persistence: keep schema validation, private permissions, atomic replacement, locking, full-field preservation, and exact-source conflicts out of QML.
• Treat QML input as untrusted: keep shared Process/Socket/Timer objects outside screen variants, validate complete protocol/state/waveform frames before one atomic assignment, retain the last valid value, and bound backend restarts.
• Preserve clipboard privacy semantics: all output remains clipboard paste; Wayland transcript and restoration payloads use sensitive hints, with no equivalent guarantee on XWayland.
• Preserve the Qt 6 shader ABI: compile .qsb, use the Qt uniform block/reflection layout, output premultiplied alpha, and keep every CI target.

Packaging

There is no distribution package recipe in packaging/; installation is implemented by the Makefile. make install:

• installs the release binary under $(PREFIX)/bin;
• compiles and installs the HUD shader, QML assets, Settings, and bundled font under $(PREFIX)/share/voice-input;
• installs the Pi extension under ~/.pi/agent/extensions;
• renders user service templates and installs desktop snippets;
• creates private config and encrypted-credential directories;
• creates a user config only when none exists and may migrate recognized older files;
• removes obsolete installed GTK/Python assets.

PREFIX does not relocate every user path. Packagers must set SYSTEMD_USER_DIR, PI_EXTENSIONS_DIR, CONFIG_HOME, and related variables or stage files explicitly. Do not package local configs, credentials, runtime state, recordings, clipboard backups, session JSONL, logs, or generated build output.

Contribution workflow

The repository contribution guide defines the Rust, QML, Quickshell, shader, resource-bound, and validation standards.

1. Use a focused topic branch and keep unrelated local configuration out of the patch.
2. Add unit tests for parsing and invariants; document reproducible manual checks for compositor, audio, provider, or QML behavior.
3. Update every public-schema surface and both documentation languages when behavior changes.
4. Run the full validation and shader commands above; inspect QSB targets/reflection and run git diff --check.
5. Inspect the patch for credentials, absolute user paths, transcripts, terminology, clipboard backups, session files, recordings, and generated artifacts.
6. In the pull request, explain compatibility, migration, privacy/data flow, resource limits, and test evidence.

See also: Architecture · Security and Privacy