Skip to content

obs(cloud): severity-mapped logs, /health, log redaction (GCP audit item 3) - #363

Merged
oratis merged 1 commit into
mainfrom
claude/gcp-observability-remediation-c3fd9e
Aug 20, 2026
Merged

obs(cloud): severity-mapped logs, /health, log redaction (GCP audit item 3)#363
oratis merged 1 commit into
mainfrom
claude/gcp-observability-remediation-c3fd9e

Conversation

@oratis

@oratis oratis commented Aug 20, 2026

Copy link
Copy Markdown
Owner

背景

2026-08-20 GCP 全项目打点盘点发现 lisa-cloud / anthropic-relay 的代码级观测缺陷(盘点报告存档于会话记忆 gcp-observability-audit.md)。本 PR 落地其中仓库内可改的部分;GCP 侧配置(uptime check、告警、通知渠道)另行按 runbook §6 执行。

改了什么

src/log.ts(新增)

  • logInfo/logWarn/logError:在 Cloud Run 上(K_SERVICE 存在)输出单行结构化 JSON,Cloud Logging 提取 severity 字段——INFO 不再落成 ERROR。本地行为不变(照旧 stderr 文本),LISA_LOG_FORMAT=json|text 可强制两个方向。
  • redactId(前 4 + 后 4)/ redactEmail(首字符+域名):日志行脱敏助手。

src/web/server.ts

  • 58 处 console.error 全部按语义归类到新 logger(此前全部落 stderr → Cloud Logging 全记 ERROR,告警没有信噪比可言)。
  • 新增 GET /health / /healthz(pre-gate、无凭据、零 I/O)——此前 runbook 只能拿 /api/auth/config 凑数。
  • 日志隐私:不再输出 idle 消息正文(原来前 120 字符)、screen-advisor 标题(来自用户屏幕截图)、reflect 摘要正文、reviewer 邮箱明文、完整 uid / 支付事务 id(Stripe/IAP 各处统一走 redactId)。

packaging/gcp-relay/index.mjs

  • 每个代理请求一行结构化请求日志(method/path/status/latency;绝不记 header 和 body——进方向有 relay token,出方向有真实 Anthropic key)。
  • 两处 catch(upstream fetch 失败、外层兜底)补错误日志——此前整个 relay 除启动行外零日志,500/502 无迹可循。

docs/RUNBOOK_CLOUD_PROD.md §6

  • 修正致命笔误:文档让告警盯大写 ANOMALY,但 meter.ts:114 实际输出 [billing] ⚠ anomaly:——照文档建的告警永远不触发。已改为照抄实际输出的过滤串。
  • 补齐:通知渠道先行(没绑渠道的告警等于没建,telloria/loopback 的现成教训)、uptime 改打 /health、新增 5xx 日志告警策略 JSON。

验证

  • npm run typechecknpm run check:api-contractnpm test 1637 pass / 0 fail ✅
  • 新增 src/log.test.ts(结构化格式 + 两个脱敏助手)。

不在本 PR 范围

  • meter.ts 异常去重仍是进程内存态(MAX_INSTANCES=3 会重复告警)——需要共享态,另行处理。
  • gateway.ts / push.ts 等其他 web 模块的 console.error 未动(同法可迁,后续小 PR)。
  • GCP 侧资源创建(uptime/告警/渠道)按更新后的 runbook §6 人工执行。

🤖 Generated with Claude Code

GCP observability audit (2026-08-20) remediation, code side:

- src/log.ts: logInfo/logWarn/logError emit structured JSON with a real
  severity on Cloud Run (K_SERVICE) so INFO stops landing as ERROR in
  Cloud Logging; plain stderr locally (unchanged Mac behavior).
  redactId/redactEmail helpers for log-line privacy.
- src/web/server.ts: all 58 console.error sites classified onto the new
  logger; GET /health(/healthz) liveness endpoint (pre-gate, no I/O);
  logs no longer carry idle-note text, screen-advisor titles, reflect
  summaries, reviewer emails, or full uid/transaction ids.
- packaging/gcp-relay/index.mjs: per-request log line (method/path/
  status/latency, never headers or bodies) + error logging in both
  catch paths — the relay was previously silent about every failure.
- docs/RUNBOOK_CLOUD_PROD.md §6: the documented alert filter said
  ANOMALY but meter.ts emits '[billing] ⚠ anomaly:' — an alert built
  from the doc would never fire. Rewritten with the actual filter,
  notification-channel-first ordering, /health uptime target, and a
  5xx log alert.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@oratis
oratis merged commit a197e32 into main Aug 20, 2026
1 check passed
oratis added a commit that referenced this pull request Aug 21, 2026
Follow-ups to #363, found while verifying that PR in production.

- redactEmail existed twice with different behavior: src/log.ts kept 1
  char via indexOf, src/web/mailer.ts:74 kept 2 via lastIndexOf and also
  handled a trailing "@". The stricter one is now the only one — it moves
  into log.ts beside the other redactors (so the dependency runs web →
  log, not the reverse) and mailer.ts re-exports it for its own callers.
- mailer.ts's four log sites were still bare console.error, so a
  successful send filed at the same level as a failed one: outcome=success
  is INFO, skipped_no_key is WARNING, both send_failed paths are ERROR.
  Recipients were already redacted; that is unchanged.
- Startup path had no severity at all on Cloud Run — cli.ts's two serve
  lines, soul/git.ts's git-missing warning, and entrypoint.sh's three
  echoes. The shell ones get a log_line helper mirroring src/log.ts:
  JSON under K_SERVICE, plain text otherwise.

Verified: /health-serving revision 00021 shows [sweep]/[orchestrator] at
INFO and reviewer email redacted, so this only covers what #363 missed.

npm test 1637 pass / 0 fail; typecheck + api-contract clean.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
oratis added a commit that referenced this pull request Aug 21, 2026
The >$10/day/user anomaly alert deduped through a process-local Set, so
with MAX_INSTANCES>1 the same day paged the operator once per instance,
and a cold start re-armed it. The Set stays as the in-process fast path;
the cross-instance arbiter is now a create-only Firestore write on
lisa-anomaly-alerts/{uid}_{day} — the second writer fails its
precondition, and that failure is the dedup signal.

Deliberately fail-open: only 409/412 counts as "someone else has it".
Any other failure (network, permission, Firestore down) still alerts,
because a duplicate alert is a nuisance while a missed one is an
unnoticed burn. With Firestore off or outside a per-uid scope (Mac
edition, shared-token demo) the claim is a no-op and behavior is
unchanged — verified by two tests that assert the network is never hit.

Also finishes this file's share of #363/#364: both console.error sites
become logError, and the alert text names a cloud tenant by redacted uid
instead of embedding the home path (which contains the raw uid). The
"[billing] ⚠ anomaly" prefix the alert policy matches is untouched, and
that policy already matches both textPayload and jsonPayload, so the
move to structured output does not break it.

npm test 1643 pass / 0 fail; typecheck + api-contract clean.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant