feat(shadow)!: mismatchLogging content controls, value projection, and diff logging - #138
feat(shadow)!: mismatchLogging content controls, value projection, and diff logging#138lan17 wants to merge 9 commits into
Conversation
…rols and log hooks
Shadow mismatch warnings are now composed field by field through the
runtime ShadowConfig.mismatchLogging group ({key, value, diff}, each
default-off, merged leaf-wise) instead of the removed all-or-nothing
logMismatches boolean. Two per-use-case hooks shape the logged content:
shadowMismatchLogValue projects both sides before value logging and the
built-in diff, and shadowMismatchLogDiff replaces the built-in diff and
receives the raw compared values. The built-in structural diff uses
microdiff over the projected-or-raw forms, oriented cached-to-source,
with non-plain-object roots collapsing to one root-level change entry.
All rendering happens eagerly at mismatch confirmation, fails closed to
null fields, and keeps the existing byte caps; raw compared values are
no longer retained until log time.
The removed shadow.logMismatches field is rejected like shadowRamp:
defaults throw at registration and stale runtime configs fail
resolution as config_error, so live configs must migrate to
shadow.mismatchLogging before adopting this release.
Covers the gaps in diff-logging coverage: CREATE entries, nested Date leaves rendering as ISO strings, index-wise array-shift noise, cyclic inputs failing closed to a null diff, a projection throw nulling the built-in diff, one projection per side feeding value and diff output together, an idle shadowMismatchLogDiff hook when diff logging is off, and hooks staying uninvoked for superseded candidates.
…ty, and hardening Resolves the accepted findings from the multi-lane review of the mismatchLogging feature: - The built-in diff now renders both sides to native JSON before diffing, so toJSON redaction and serializer normalization bound the diff exactly as they bound value logging: no more leaking fields that toJSON hides, no phantom entries for serializer-normalized Dates, and mixed object/array roots collapse to one root-level change entry as documented. Identical loggable forms short-circuit to []. - diffJson has one cap owner: previewShadowLogJson takes a byte budget, the hook path and built-in path both clamp with the diff cap, and previewShadowLogDiff delegates instead of duplicating the body. - Runtime shadow config is read by own properties only (group, leaves, and ramp, in both the merge and admission reads), so prototype-carried values can neither enable payload logging nor admit shadow work. - A non-object runtime mismatchLogging group is documented as malformed config shape that fails resolution as config_error, matching the layer-map precedent; the unreachable admission-time branch is deleted and the behavior pinned by runtime-overlay tests alongside the previously uncovered removed-logMismatches rejection. - The leaf set is derived from one exhaustive, compile-checked list; ShadowLogPlan aliases Required<ShadowMismatchLoggingConfig>; the disabled() kill-switch literal is annotated exhaustive. - Dead emit-time fallback deleted: every preview fails closed to a null field (previewShadowLogKey included), the warning path is throw-free by construction, and the warning payload is built fresh so a mutating metrics adapter cannot contaminate it. - New tests: both-hooks projection/raw split, prototype pollution, Object.create-carried leaves, per-field fail-closed, toJSON-bounded diff, serializer-normalization phantom, mixed-kind roots, explicit byte budgets, and the runtime rejection rows.
lan17
left a comment
There was a problem hiding this comment.
Blocking review: I found six issues that should be addressed before merge. Five are attached inline.
The remaining release blocker is:
[P1] Add an exact BREAKING CHANGE: footer to the PR body. The migration heading is useful to human readers, but this repository's conventionalcommits release generator only recognizes the footer form called for in README.md. Because squash commits use the PR body, the generated release notes will otherwise omit the breaking-change section. That is especially risky here: stale shadow.logMismatches runtime config fails resolution and bypasses caching for each affected invocation, so operators need the config-before-code deployment order surfaced in release notes.
All seven checks are green, and I also ran the full local check successfully (typecheck, 551 tests, build/declarations, and packed ESM/CJS package validation). The exact-head CI integration run passed 139 Redis/Valkey tests. The targeted cases in the inline findings are not covered by those checks.
…and render-once own-key diff Review fixes for PR #138: - The shadow group itself is now an own-property read at the constructor, defaults-snapshot, and runtime-merge boundaries, so a prototype-inherited group can no longer activate logging policy (its leaves are own properties and passed every inner gate). - Unknown mismatchLogging fields fail closed: defaults reject them at registration, and a runtime override carrying one (a typo'd emergency shutoff like `vaule: false`) turns the whole logging group off with one config_resolution error instead of silently inheriting enabled leaves. The merge carries unknown own keys through so admission can see them. - Mismatch warnings render each side to native JSON exactly once and derive the value fields and the built-in diff from the same snapshot, so a stateful toJSON cannot put data in diffJson that value logging redacts. A side with no JSON rendering (top-level undefined, cycles, bigint, a thrown hook or projection) fails the diff closed to null instead of emitting a self-contradictory root entry. - The built-in diff is now a small own-key differ over the parsed JSON snapshots, replacing microdiff: prototype-carried data (enumerable Object.prototype or Array.prototype pollution) can never reach diffJson, and the microdiff runtime dependency is removed. Entry format, orientation, and array index-wise semantics are unchanged and remain pinned by tests.
…and render-once own-key diff Review fixes for PR #138: - The shadow group itself is now an own-property read at the constructor, defaults-snapshot, and runtime-merge boundaries, so a prototype-inherited group can no longer activate logging policy (its leaves are own properties and passed every inner gate). - Unknown mismatchLogging fields fail closed: defaults reject them at registration, and a runtime override carrying one (a typo'd emergency shutoff like `vaule: false`) turns the whole logging group off with one config_resolution error instead of silently inheriting enabled leaves. The merge carries unknown own keys through so admission can see them. - Mismatch warnings render each side to native JSON exactly once and derive the value fields and the built-in diff from the same snapshot, so a stateful toJSON cannot put data in diffJson that value logging redacts. A side with no JSON rendering (top-level undefined, cycles, bigint, a thrown hook or projection) fails the diff closed to null instead of emitting a self-contradictory root entry. - The built-in diff is now a small own-key differ over the parsed JSON snapshots, replacing microdiff: prototype-carried data (enumerable Object.prototype or Array.prototype pollution) can never reach diffJson, and the microdiff runtime dependency is removed. Entry format, orientation, and array index-wise semantics are unchanged and remain pinned by tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lan17
left a comment
There was a problem hiding this comment.
One blocking finding remains after the follow-up commit. The other five findings from the prior review are fixed, and the exact-head local/CI validation is green, but the prototype-data boundary for the built-in diff is still incomplete. Details are attached inline.
…N hooks The own-key differ kept prototype data out of the entries, but the finished entry tree was still handed to native JSON.stringify, whose inherited-toJSON lookup let a polluted or legacy Array.prototype.toJSON replace the whole diff. The diff tree is now serialized by a closed-domain walker that only gives primitives to native JSON, so toJSON runs solely while rendering user data into the side snapshots, never over the internally generated entries.
1699bbb to
7aa731c
Compare
lan17
left a comment
There was a problem hiding this comment.
Correction to my previous review: I withdraw the remaining inherited-prototype blocker. It applied a hostile-intrinsics threat model that does not match this feature’s trusted-value boundary.
The current head now keeps the design deliberately simple: one native JSON snapshot per side, a private parsed JsonValue, own-member structural traversal, and ordinary bounded JSON serialization. The custom prototype-hardening code, tests, and claims have been removed.
The other five findings remain addressed. Full local validation passed (typecheck, 560 tests, build/declarations, and packed ESM/CJS package checks); Redis integration passed 137 tests with the two GLIDE Cluster cases skipped by the local Docker environment. All current GitHub checks pass. I have no remaining findings from this review.
Every opted-in mismatch warning now carries cachedValueAgeSeconds, the same coarse mixed-clock age observeShadowValueAge records for the verdict, so a single log line distinguishes a seconds-old race from a days-old invalidation bug without consulting the histogram.
Treat promise-like logging-hook results as unavailable and consume their settlements so detached shadow work cannot leak unhandled rejections. Preserve unknown runtime logging keys even when explicitly undefined so closed-schema admission rejects the whole group.
Apply known config fields while ignoring unknown own keys across defaults and runtime overlays, including legacy shadow fields. Emit one bounded config_unknown_field error label per observed config without exposing field names or values.
Why
Shadow validation compares a cached value against the source of truth and emits a
mismatchmetric when they disagree. Until now the only way to see what disagreed wasshadow.logMismatches: true, which logged the cache key plus the full JSON of both values — all or nothing. That's a problem for use cases whose values carry user data: one dynamic-config flip could put whole records into logs.This PR splits the warning into parts you opt into individually at runtime, and adds two per-use-case hooks so values can be redacted (or summarized as a diff) before they ever reach the logger.
The new runtime config
ShadowConfig.logMismatchesis gone. In its place:keycacheKey— the logical DialCache URNvaluecachedValueJson+sourceValueJsondiffdiffJson— which paths differ and howSemantics worth knowing:
cacheNamespace/useCase/keyType/outcomefor routing, pluscachedValueAgeSeconds— how long the stale value had been readable when validation caught it (the same age theobserveShadowValueAgemetric records).valueoff mid-incident while keepingkeyon — no deploy needed.value: "yes"from a bad config push) acts asfalseand records oneconfig_resolutionerror; valid sibling fields keep logging. The cache result is never affected.error=config_unknown_fieldmetric; field names and values never become labels. This includes legacyshadowRampandshadow.logMismatchesfrom mixed-version providers.DialCacheKeyConfig.disabled()sets all three to false, so the kill switch still kills logging.Shaping what gets logged (code-level hooks)
Two optional hooks live next to
shadowComparatoroncached()/getOrLoad():value: trueanddiff: truelog the raw values, same material as the old behavior. Define the projector for any use case whose values can carry sensitive fields before enabling those flags.nullfor that side (and anullbuilt-in diff) — it never falls back to raw values. A diff-hook throw or promise-like result logsdiffJson: null. Promise-like settlements are consumed but never awaited, so an accidentally async callback cannot leak an unhandled rejection or silently create a second hook contract.What a warning looks like
The built-in diff
No new dependencies: the built-in diff is a small own-key differ over the native-JSON forms of both sides — each side is rendered once and the value fields and diff derive from the same snapshot, so
toJSONredaction and serializer normalization bound the diff exactly as they bound value logging. Entries are{ type: "CREATE" | "REMOVE" | "CHANGE", path, value, oldValue }, oriented cached → source (oldValueis what Redis had). A side with no JSON rendering (top-levelundefined, cycles,bigint, or a thrown or promise-like hook result) fails the diff closed tonull.toJSONredaction and serializer normalization bound the diff exactly as they bound value logging — no leaking fieldstoJSONhides, and no phantom entries when a deserialized ISO string meets a liveDateof the same instant.diffJson: "[]"means the loggable inputs held no visible difference — with a projector, that reads as "the difference is inside fields you chose not to log", which is itself a useful signal.Migration — breaking API, compatible runtime
shadow: { logMismatches: true }shadow: { mismatchLogging: { key: true, value: true } }shadow: { logMismatches: false }mismatchLogging(or set fields tofalse)shadow.logMismatchesis no longer read, but older default or runtime config carrying it does not break cache resolution. It is ignored like any unknown field, recognized fields still apply, and DialCache emits the boundedconfig_unknown_fieldsignal. The oldtruevalue therefore does not enable mismatch logging; migrate tomismatchLoggingto retain that behavior.Review
A six-lane review (correctness, tests, simplicity, architecture, contracts, security, plus a two-stage holistic audit) ran against the feature; all 10 accepted findings are fixed in the follow-up commit: diff-input normalization (the
toJSON/serializer asymmetries above), one cap owner fordiffJson, own-property-only runtime shadow config reads (prototype pollution andObject.create-carried leaves can neither enable payload logging nor admit shadow work), documented-and-pinnedconfig_errorsemantics for a malformedmismatchLogginggroup (matching the layer-map precedent), an exhaustive compile-checked leaf list, and removal of the dead emit-time fallback (every warning field now fails closed tonullindependently).A follow-up correctness and simplicity review closed the async-hook rejection leak. A later compatibility pass made unknown key-config fields permissive across defaults and runtime overlays: normalization keeps only recognized fields, one bounded metric reports each affected config, and known malformed values retain their existing validation.
Testing
pnpm typecheck·pnpm test(559 passing, ~97.7% coverage) ·pnpm build·pnpm test:package(packed ESM + CJS consumers) · CIpnpm test:integration(all 139 Redis and cluster tests passing).null, projection failure nulling the built-in diff, one projection per side feeding value + diff together, custom diff hook receiving raw values (and staying idle whendiffis off), promise-like custom-diff results failing closed without unhandled rejection, hooks staying idle onsuperseded, per-leaf invalid runtime config, unknown fields at every key-config scope (including explicitundefinedand both legacy shadow fields) being ignored with one bounded metric while recognized fields remain effective, leaf-wise merge, and clone/freeze.CREATEentries, nestedDate→ ISO strings, array-shift noise, mixed-kind nodes, unrenderable sides and cyclic inputs failing closed tonull, own-member traversal, and single-rendertoJSONconsistency.🤖 Generated with Claude Code
BREAKING CHANGE: ShadowConfig.logMismatches was replaced by the shadow.mismatchLogging content group ({ key, value, diff }); the previous logMismatches: true behavior is mismatchLogging: { key: true, value: true }. Legacy logMismatches fields are ignored at runtime and emit config_unknown_field rather than failing cache resolution, but they no longer enable logging, so migrate config stores to retain that behavior.