feat(telemetry): activation + reliability signals (schema v8)#181
Merged
Conversation
Adds anonymous, opt-out telemetry to close the activation funnel and make call failures diagnosable. All additive (schema v7->8), Python<->TS parity, fire-and-forget (never affects a call), no PII. - config_incomplete event: emitted once when required runtime config is missing (missing=carrier_credentials|llm_key|engine_config), immediately before the startup raise — the activation-blocker signal. - call_completed per-stage latency: stt_latency_ms, llm_ttft_ms, tts_first_byte_ms, eou_latency_ms (whole ms, omitted when a stage didn't run). - call_completed error taxonomy: error_layer + disconnect_reason enums. - time_to_first_call_bucket on call_started/call_completed. - client sampling: PATTER_TELEMETRY_SAMPLE (default 1.0), deterministic per run, never samples first_run/config_incomplete/error; sample_rate weight stamped for extrapolation. Tests: telemetry Py 116 / TS 87 in new dedicated files; full suites green; lint + build clean; Py<->TS parity verified.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v7 → v8.Implementation
config_incompleteevent — emitted once when required runtime config is missing, immediately before the existing startupraise(control flow unchanged). Dimensionmissing ∈ {carrier_credentials, llm_key, engine_config, other}— never the key value or any PII. This is the activation-blocker signal (installs that run but can't proceed).call_completed:stt_latency_ms,llm_ttft_ms,tts_first_byte_ms,eou_latency_ms(whole ms, read off the existing CallMetrics breakdown; a dimension is omitted when its stage didn't run, e.g. Realtime).call_completed:error_layer ∈ {stt,llm,tts,carrier,config,internal,none,other}+disconnect_reason ∈ {hangup_local,hangup_remote,error,timeout,no_answer,busy,completed,other}, derived from the already-available outcome/error_code (no new call-path plumbing).time_to_first_call_bucketoncall_started/call_completed(install-age at call time).PATTER_TELEMETRY_SAMPLE(default1.0= no sampling), deterministic per run via the run id hash, never samplesfirst_run/config_incomplete/error events; the chosensample_rateis stamped so analytics can extrapolate (weight = 1/sample_rate).libraries/{python/getpatter,typescript/src}/telemetry/{events,client,env,install_id,call_metrics}.*+client.*. New dedicated test files per feature (the shared telemetry test files are only touched for theSCHEMA_VERSIONbump).## Unreleased; the bump +Unreleased → X.Y.Zrename happen at release time per the release workflow.Breaking change?
No. Every addition is optional with a safe default (
PATTER_TELEMETRY_SAMPLEdefaults to 1.0), no existing event/dimension changed or removed, and the schema bump is purely additive. Existing callers are unaffected.Test plan
pytest tests/— telemetry 116 new/updated pass; full suite 2712 pass / 0 failnpm test(telemetry 87 pass; full 2142 pass / 0 fail) +npm run lint(clean) +npm run build(ok)SCHEMA_VERSION=8, defaults all identical)Docs updates
CHANGELOG.mdupdated (## Unreleased→### Addedfor the new signals,### Changedfor the schema bump).docs/telemetry.mdxto documentPATTER_TELEMETRY_SAMPLE, theconfig_incompleteevent, and the newcall_completeddimensions; feature-inventory rows.