obs(billing): dedupe anomaly alerts across instances - #366
Merged
Conversation
The >$10/day/user anomaly alert deduped through a process-local Set, so
with MAX_INSTANCES>1 the same day paged the operator once per instance,
and a cold start re-armed it. The Set stays as the in-process fast path;
the cross-instance arbiter is now a create-only Firestore write on
lisa-anomaly-alerts/{uid}_{day} — the second writer fails its
precondition, and that failure is the dedup signal.
Deliberately fail-open: only 409/412 counts as "someone else has it".
Any other failure (network, permission, Firestore down) still alerts,
because a duplicate alert is a nuisance while a missed one is an
unnoticed burn. With Firestore off or outside a per-uid scope (Mac
edition, shared-token demo) the claim is a no-op and behavior is
unchanged — verified by two tests that assert the network is never hit.
Also finishes this file's share of #363/#364: both console.error sites
become logError, and the alert text names a cloud tenant by redacted uid
instead of embedding the home path (which contains the raw uid). The
"[billing] ⚠ anomaly" prefix the alert policy matches is untouched, and
that policy already matches both textPayload and jsonPayload, so the
move to structured output does not break it.
npm test 1643 pass / 0 fail; typecheck + api-contract clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GCP 打点收尾的最后一个代码项（#363 → #364 → 本 PR）。
问题
>$10/天/用户的异常消费告警靠一个进程内 Set 去重：MAX_INSTANCES=3时同一天会告警最多 3 次，而且冷启动就重新武装 —— 一次流量波动导致的多实例扩容，运维会收到一串重复推送。做法
Set 保留为进程内快速路径；跨实例的仲裁改成
lisa-anomaly-alerts/{uid}_{day}上的 create-only 写（setDoc(..., { exists: false })）——第二个写入者的 precondition 失败就是去重信号。这个模式直接沿用iap.ts已有的全局 transactionId 索引（同一个firestore.ts原语）。刻意 fail-open：只有
409/412才判定为"别人已认领"。其它任何失败（网络、权限、Firestore 挂了）一律照常告警 —— 重复告警只是噪音，漏告警是一笔没人注意到的持续烧钱。Firestore 关闭或不在 per-uid scope 内（Mac 版、共享 token demo）时 claim 是 no-op，行为完全不变 —— 有两个测试断言这两种情况下网络一次都不会被碰。
顺带收尾本文件的日志债
console.error→logError（obs(cloud): severity-mapped logs, /health, log redaction (GCP audit item 3) #363/obs(cloud): dedupe redactEmail, classify mailer + startup logs #364 漏掉的最后一个文件，现在src/billing/零裸 console）。lisaHome()，云端该路径形如/data/users/<uid>，等于把原始 uid 写进日志和推送。改为uid ${redactId(uid)}；Mac 版仍显示本地路径（无 uid，且对本地用户更有意义）。告警不会因此失效（已核实）：策略匹配的是
[billing] ⚠ anomaly前缀，未改动；且该策略的过滤串已是双写法——所以从 textPayload 转到 jsonPayload 两边都命中。
验证
npm test1643 pass / 0 fail（新增 6 个 claim 测试）；typecheck、api-contract 干净。运维说明
lisa-anomaly-alerts是新集合，每个触发阈值的用户每天一个小文档。正常情况下几乎不产生（超阈值本就是异常事件），无需清理策略；真要收也可以按day字段配 Firestore TTL。🤖 Generated with Claude Code