diff --git a/AGENTS.md b/AGENTS.md index acc2626..7572450 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,11 +2,13 @@ ## Project overview -DialCache is a TypeScript caching library with explicit request-scoped enablement, local and Redis layers, runtime rollout controls, request coalescing, targeted invalidation, and adapter-based observability. +DialCache is a TypeScript caching library with explicit request-scoped enablement, local and Redis layers, runtime rollout controls, request coalescing, detached Redis shadow validation and bootstrap, targeted invalidation, and adapter-based observability. ## Structure ```text +README.md # Adoption guide, safety model, and reference routing +docs/ # Focused user-facing configuration and operations guides src/ index.ts # Public root entry point (barrel) dialcache.ts # Main DialCache API and cached-function wrapper @@ -33,9 +35,21 @@ test/ # Unit and Redis integration tests - Active same-key work is coalesced before the first active cache layer, using request scope for request-local caching and process scope for shared layers, unless the use case's resolved `coalesce` policy disables it. +- Shadow validation is opt-in, detached work for tracked and untracked remote + caches that never supplies or delays the caller; it has separate sampling, + capacity, deadline, invalidation, and observability contracts. +- Shadow policy is grouped under `DialCacheKeyConfig.shadow`. Mismatch logging + is default-off diagnostic output; its size limits are not redaction. - Cache plumbing fails open; explicit maintenance operations surface mutation failures. +- `invalidateRemote()` requires a configured Redis client and rejects when the + client is absent or the watermark mutation fails. - Tracked Redis values and invalidation watermarks share a Redis Cluster hash tag. -- Tracked reads run on primaries so replica lag cannot hide invalidation. +- Tracked reads require one authoritative value/watermark snapshot; cluster + adapters explicitly route them to primaries so replica lag cannot hide + invalidation. +- Redis payload compression is default-on for writes, while reads always + interpret the compression envelope so disabling new compression does not + strand existing entries. - A tracked write's placeholder frame (version byte 0) is unreadable on both read paths until the stamp script promotes it, and the stamp promotes only the placeholder carrying its own per-write nonce. @@ -45,8 +59,13 @@ test/ # Unit and Redis integration tests ## Conventions - Preserve strict TypeScript settings and public abstraction boundaries. +- Keep the README focused on evaluation and adoption. Put complete operational + contracts in a focused `docs/` guide and link it from the relevant README + summary. - Keep Redis client-specific behavior in adapters; core code depends on `DialCacheRedisClient`. -- Public exports belong in the root or an explicit integration entry point such as `src/node-redis.ts`, `src/prometheus.ts`, or `src/redis-protocol.ts`. +- Public exports belong in the root or an explicit integration entry point such + as `src/node-redis.ts`, `src/valkey-glide.ts`, `src/prometheus.ts`, + `src/datadog.ts`, or `src/redis-protocol.ts`. - Use `corepack pnpm` for project commands. ## Validation diff --git a/README.md b/README.md index 8d85510..12e8006 100644 --- a/README.md +++ b/README.md @@ -4,50 +4,78 @@ [![Codecov](https://codecov.io/gh/lan17/DialCache/branch/main/graph/badge.svg)](https://codecov.io/gh/lan17/DialCache) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/lan17/DialCache/badge)](https://scorecard.dev/viewer/?uri=github.com/lan17/DialCache) -Fine-grained TypeScript caching with explicit enabled contexts, request-local memoization, process-local and Redis TTL caching, stable key construction, runtime rollout controls, request coalescing, adapter-based observability, and Redis watermark-based targeted invalidation. +**Read-through caching with the controls production systems need.** + +DialCache is a TypeScript read-through caching library for database and service +reads in production Node.js applications. + +Wrap a reusable function with `cached()` or keep a loader inline with +`getOrLoad()`; when the active cache layers miss, DialCache calls your loader +and publishes the result to whichever request-local, bounded process-local, +and optional Redis or Valkey layers are active. + +Around that core path, DialCache provides patterns that high-scale services +otherwise have to build themselves: request coalescing, per-use-case runtime +policy, deterministic ramp-up and ramp-down, detached shadow validation, +fail-open cache access, targeted invalidation, transparent Redis compression, +deadlines, and backend-neutral metrics. + +The “dial” is the runtime policy: start a use case at zero, expand local or +remote caching to stable key cohorts, and reverse the rollout without changing +the loader. + +DialCache is a backend application library—not a frontend data cache, cache +server, Redis or Valkey client, or configuration control plane. Your service +owns the loader, clients, dynamic configuration source, cache identity, TTLs, +invalidation windows, admission control, and resource budgets. + +## Safety comes from explicit controls + +- **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the + loader without building a key, resolving policy, accessing a cache, or + coalescing work. +- **Gradual and reversible.** Start process-local and remote ramps at `0`, + expand either to a stable key cohort, and turn every cache layer back off + through runtime policy. +- **Fail-open cache path.** Cache-plumbing failures fall through to the loader + instead of replacing a usable result. Explicit invalidation failures still + surface to the caller. ## Contents - [Install](#install) - [Quick start](#quick-start) -- [How caching works](#how-caching-works) -- [Enabled context](#enabled-context) -- [Defining cached functions](#defining-cached-functions) - - [One-shot inline cache blocks](#one-shot-inline-cache-blocks) -- [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions) -- [Runtime config and ramp controls](#runtime-config-and-ramp-controls) -- [Cache layers](#cache-layers) - - [Request-local cache](#request-local-cache) · [Process-local cache](#process-local-cache) · [Redis-backed TTL cache](#redis-backed-ttl-cache) · [Remote read deadlines](#remote-read-deadlines-and-async-liveness) · [Serialization](#serialization) · [Compression](#compression) · [Shadow validation](#shadow-validation) -- [Cached-value ownership](#cached-value-ownership) -- [Targeted invalidation and watermarks](#targeted-invalidation-and-watermarks) -- [Request coalescing](#request-coalescing) - - [Fallback deadlines](#fallback-deadlines) · [Coalescing state](#coalescing-state) -- [Metrics](#metrics) -- [Maintainers](#maintainers) +- [Dial caching up or down](#dial-caching-up-or-down) +- [Validate Redis before serving it](#validate-redis-before-serving-it) +- [How the read path works](#how-the-read-path-works) +- [Core concepts](#core-concepts) +- [Production checklist](#production-checklist) +- [Reference guides](#reference-guides) ## Install ```bash -pnpm add dialcache -# Choose a Redis client when using the remote layer: -pnpm add redis@~4.7.1 -# or -pnpm add @valkey/valkey-glide@^2.0.0 -# Add a metrics client only when using its adapter: -pnpm add prom-client@^15.1.3 -# or -pnpm add hot-shots@^17.0.0 +npm install dialcache ``` -DialCache requires Node.js with zstd support in `node:zlib`: 22.15.0 or newer -within the 22.x line, or 23.8.0 and newer (23.0–23.7 lack zstd and are -excluded). Production deployments should use a +DialCache requires Node.js `>=22.15.0 <23.0.0 || >=23.8.0`; Node.js 23.0 +through 23.7 lack the `node:zlib` zstd API. Production deployments should use a [currently supported LTS release](https://nodejs.org/en/about/previous-releases). +Redis, Valkey, Prometheus, and Datadog integrations are optional and keep their +clients application-owned: + +- [Redis and Valkey setup](https://github.com/lan17/DialCache/blob/main/docs/redis.md) +- [Prometheus and Datadog setup](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + ## Quick start +Most services create one long-lived `DialCache` instance and reuse it across +the process. It owns one process-local LRU and one process-coalescing scope; +create separate instances only when those resources should be isolated: + ```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; const dialcache = new DialCache(); @@ -57,462 +85,88 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + }), }, ); -// Caching is OFF outside an enable() scope (see "Enabled context"), so this runs the fn uncached: +// Outside enable(), this is a true pass-through to db.fetchUser: await getUser("123"); -// Inside enable(), reads are cached: -const user = await dialcache.enable(() => getUser("123")); -``` - -## How caching works - -The wrapped function is the **fallback**: it runs whenever no active cache layer returns a value, whether because layers missed, were disabled, or failed open. - -When caching is enabled, reads flow through: - -```text -request-local cache -> process-local cache -> Redis cache -> fallback function +// Inside enable(), the first call loads and the second reuses the cached value: +const user = await dialcache.enable(async () => { + await getUser("123"); // db.fetchUser, then populate process-local cache + return await getUser("123"); // process-local hit +}); ``` -- Request-local hits return the value memoized in the current outermost `enable()` scope. -- Results from the lower chain are memoized request-locally when that layer is enabled. -- Process-local hits return immediately. -- Process-local misses try Redis and populate the process-local cache on a Redis hit. -- Redis misses call the fallback and attempt to populate Redis and, when active, the process-local cache. Tracked invalidation may suppress both publications. -- Selected Redis keys can execute non-serving [shadow work](#shadow-validation) that validates hits and fills clean misses, even before Redis is allowed to serve callers. -- Redis read failures and timeouts are logged, counted in metrics, and fail open without attempting a second Redis operation. Redis write failures also fail open. `invalidateRemote` requires a configured Redis client; missing configuration and Redis failures are logged, counted, and rethrown so callers do not assume invalidation succeeded. -- Cache-key construction and config-provider failures also fail open and run the fallback uncached. -- A missing effective process-local/Redis TTL disables that layer by policy; a configured TTL with no ramp defaults to 100%. Disabled layers record a disabled reason and fall through to the next layer/fallback. - -Caching as a whole is only active inside an enabled context, described next. - -## Enabled context +`cached(fn, options)` preserves the function's parameters and returns a +Promise-based wrapper. The configuration above enables only the process-local +layer with a 60-second TTL; its omitted ramp defaults to `100`. Request-local +memoization and the remote layer remain off. -Caching is **off by default** and only active inside a `dialcache.enable(...)` scope. This is deliberate: it lets you turn caching **off in write paths** so a stale read can't be cached around a write. DialCache uses Node `AsyncLocalStorage` to keep enabled state scoped to the current asynchronous call chain. +For a one-shot calculation that should remain inline, +[`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) +accepts a +zero-argument loader and a direct key through the same cache contract. -**Enable once at your request boundary** (e.g. a middleware that wraps read-request handling) so individual call sites don't each need it; wrap mutation handlers in `disable()`: +Enable caching once at a read-request boundary instead of at every call site. +Keep nested mutation work uncached with `disable()`: ```ts await dialcache.enable(async () => { - await getUser("123"); // cached + const user = await getUser("123"); await dialcache.disable(async () => { - await updateUser("123", patch); // reads here are uncached + await updateUser("123", patch); }); - - await getUser("123"); // cached again }); ``` -- Default is disabled — `cached()` and `getOrLoad()` calls made **outside** any `enable()` scope simply run their loader uncached (no error), so wrap your read paths to actually cache. -- Enabled state is async-scope-local, not process-global. -- Nested `enable` / `disable` scopes restore the previous behavior when the callback completes. Nested `enable()` calls reuse the outer request-local scope rather than creating a new one. - -## Defining cached functions - -Use `cached(fn, options)` for an extracted, reusable function. The wrapped callable has the same parameters and always returns a `Promise`. For a one-shot calculation that should remain inline, use [`getOrLoad()`](#one-shot-inline-cache-blocks). - -| Option | Required | Description | -| --- | --- | --- | -| `keyType` | yes | The kind of id the key addresses (e.g. `"user_id"`). Together with the id, the invalidation unit for tracked entries. | -| `useCase` | yes | Identifies the individual cache: part of the stored key and the metrics label. | -| `cacheKey` | yes | Selector over `fn`'s parameters; returns a bare id or `{ id, args }`. | -| `defaultConfig` | no | `DialCacheKeyConfig` baseline policy that runtime config overlays field by field (see [Runtime config](#runtime-config-and-ramp-controls)). | -| `serializer` | when the return type is not statically JSON-compatible | Per-function `Serializer` for Redis values (see [Serialization](#serialization)). | -| `shadowComparator` | no | Synchronous application-level equality for shadow validation; defaults to Node's strict deep equality. | -| `trackForInvalidation` | no (default `false`) | Opts this use case's Redis entries into watermark-based targeted invalidation. | -| `fallbackTimeoutMs` | no (default `60_000`) | Fallback deadline in milliseconds, at most 2,147,483,647; `null` disables it (see [Fallback deadlines](#fallback-deadlines)). | - -`cached()` validates `useCase` at registration: a duplicate within one `DialCache` instance throws `UseCaseIsAlreadyRegisteredError`. Both APIs reject the internal name `watermark` with `UseCaseNameIsReservedError`. - -### One-shot inline cache blocks - -`getOrLoad(load, options)` runs one zero-argument loader through the same policy, cache layers, coalescing, invalidation, metrics, serialization, deadlines, and fail-open behavior as `cached()`. It is useful when only part of a larger function should be cached and the loader needs to capture local values: - -```ts -// Reuse the caller-owned defaults; getOrLoad() snapshots them per invocation. -const profileCacheDefaults = DialCacheKeyConfig.enabled(60); - -const profile = await dialcache.getOrLoad( - async () => { - const user = await db.getUser(userId); - return renderProfile(user, locale); - }, - { - keyType: "user_id", - useCase: "BuildProfile", - key: { id: userId, args: { locale } }, - defaultConfig: profileCacheDefaults, - }, -); -``` - -The options match `cached()` except that the direct `key` replaces the `cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and snapshotted for each invocation. Outside an enabled scope, DialCache invokes `load` directly without constructing a key or resolving runtime policy. +Enabled state follows the current asynchronous call chain through Node +`AsyncLocalStorage`; it is not process-global. Nested scopes restore the +previous state when their callbacks settle. -`getOrLoad()` does not register its `useCase`, so repeated calls should reuse one stable, deployment-defined name such as `"BuildProfile"`. Keep it bounded: never derive `useCase` from a user, request, id, or other high-cardinality input because it is part of both cache identity and metrics labels. Put those values in `key` instead. +`disable()` prevents cache access during its callback; it does not evict values +cached before a mutation. Use the appropriate invalidation or TTL policy before +serving later reads of mutable data. -Every captured value that can change the result belongs in the bare id or `{ id, args }` key. Concurrent same-key calls may share one caller's in-flight loader and cached value, so all call sites for that identity must also agree on value meaning and serialization. Prefer `cached()` when a loader is reusable; prefer `getOrLoad()` when the calculation is intentionally local to one call site. +### From local trial to production -## Keys, ids, and extra dimensions +A typical adoption path treats the quick start as local verification, not as +the initial production rollout policy: -For `cached()`, the key comes from the required `cacheKey` selector whose parameters are inferred from `fn`. `getOrLoad()` accepts the same bare id or `{ id, args }` shape directly through `key`: +1. verify the loader, key, and process-local behavior in development; +2. before production, add + [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + and an application-owned runtime policy that sets both shared serving ramps + and `shadow.ramp` to `0`; +3. add a remote TTL and + [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) + while the remote serving ramp remains `0`; +4. optionally + [validate and fill Redis in shadow mode](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) + without serving it; and +5. increase process-local, shadow, and remote cohorts independently while + monitoring their load and outcomes. -The selected or direct key is the value identity contract. It must include every input dimension that can affect the returned value; otherwise distinct calls can reuse the same cached value or share the same in-flight fallback through default-on request coalescing. +## Dial caching up or down -```ts -const searchPosts = dialcache.cached( - (userId: string, page: number, filter: string) => db.searchPosts(userId, page, filter), - { - keyType: "user_id", - useCase: "SearchPosts", - cacheKey: (userId, page, filter) => ({ id: userId, args: { page, filter } }), - defaultConfig: DialCacheKeyConfig.enabled(60), - }, -); -await dialcache.enable(() => searchPosts("u1", 2, "active")); -``` +Every cache operation can declare a stable `defaultConfig`. An optional +`cacheConfigProvider` returns a sparse runtime overlay for the current key, so +policy can change independently of the loader. -`DialCacheConfig.namespace` is the logical cache namespace and the first component of every key. It defaults to `"urn"`, producing keys such as `urn:user_id:123#GetUser`. Set a stable application-specific value when multiple applications may use the same Redis deployment: - -```ts -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, -}); -``` - -That produces Redis keys beginning with `users-api:...`, or `{users-api:...}` for invalidation-tracked values. `namespace` is DialCache's single cache-identity and key-partitioning setting: it participates in request-local, process-local, Redis, coalescing, deterministic ramp, invalidation, and metrics. It may not contain `{` or `}` because DialCache reserves those characters for Redis Cluster hash tags. Use a namespace to express any required application or environment separation, such as `production-users-api`. - -- **`keyType` + `id` is the invalidation unit for tracked Redis entries.** `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one watermark for that user; any `trackForInvalidation` Redis entry with the same `keyType` and `id` is refreshed across all `args` variants when Redis is read. `invalidateRemote` does not evict existing request-local or process-local entries (see [Targeted invalidation](#targeted-invalidation-and-watermarks)), and untracked Redis entries do not consult the watermark. `useCase` identifies the individual cache (it's the metrics label and part of the stored key). -- **`args` are part of the cache key** — different `args` produce different entries — but invalidation is by `id` only. -- **Scalar key equality is string-based.** Runtime type is not an identity dimension: for matching surrounding dimensions, numeric `1`, string `"1"`, and bigint `1n` identify the same key; argument values `null` and `"null"` also match. `-0` matches `0`, and an `undefined` argument is omitted. If a deployment changes the logical meaning represented by a scalar, change an explicit identity dimension such as `keyType`, `useCase`, or an argument name/value. -- **Non-key inputs** (for example a db handle) are parameters ignored by a `cacheKey` selector or values captured by a `getOrLoad()` loader. They still reach non-coalesced executions, but concurrent same-key cache misses share the leader's execution unless the use case disables coalescing, so do not omit values like auth context, locale, or cancellation behavior unless sharing one result is correct. -- **Methods:** pass `obj.method.bind(obj)` (or `(...a) => obj.method(...a)`) — a bare `obj.method` reference loses `this`. - -Changing the namespace value intentionally creates a cold-cache boundary across every layer. Old and new keyspaces do not share Redis values or invalidation watermarks. During an overlapping deployment, an invalidation handled by one version is invisible to the other, which can continue serving a stale tracked value until its value TTL expires. If remote invalidation correctness matters, a normal rolling deployment is unsafe: use a coordinated no-overlap cutover, or an operational bridge that prevents both versions from serving remote cache across mutations (for example, temporarily disable and clear remote caching during the transition). After the cutover, provision for fallback/refill load and allow old Redis keys to expire by TTL. - -## Runtime config and ramp controls - -Instance-wide behavior is set through the `DialCache` constructor: - -| `DialCacheConfig` option | Default | Description | -| --- | --- | --- | -| `namespace` | `"urn"` | Logical cache namespace and first key component (see [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions)). | -| `redis` | none | `{ client: DialCacheRedisClient, readTimeoutMs?: number, serializer?: Serializer, compression?: CompressionConfig \| false }`; enables the Redis layer with a 50 ms default read deadline, an optional instance-default serializer, and default-on zstd payload compression (see [Redis-backed TTL cache](#redis-backed-ttl-cache), [Serialization](#serialization), and [Compression](#compression)). | -| `localMaxSize` | `10_000` | Global process-local entry cap; `0` disables process-local storage. Nonnegative safe integer. | -| `shadowMaxInFlight` | `1` | Maximum scheduled or active shadow jobs per `DialCache` instance, including uncancellable underlying work. Positive safe integer; excess work is dropped without queuing. | -| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the function's `defaultConfig`; `null` applies no overrides. | -| `metrics` | disabled | A `DialCacheMetricsAdapter` (see [Metrics](#metrics)). | -| `logger` | `console` | Receives operational cache failures and opted-in confirmed shadow mismatch warnings (`debug`, `warn`, `error`). Synchronous throws and rejections from returned promises or thenables are isolated without being awaited. | - -Per-invocation cache policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` maps keyed by `CacheLayer.LOCAL` (process-local) and `CacheLayer.REMOTE` (Redis), a `requestLocal` boolean, a `coalesce` boolean (see [Request coalescing](#request-coalescing)), an optional `remoteReadTimeoutMs`, and an optional `shadow` group. `ShadowConfig` contains the independent shadow `ramp` percentage plus the default-off `logMismatches` control. - -Every cached definition or `getOrLoad()` invocation can provide an optional per-use-case `defaultConfig`. It is the baseline policy, and the `cacheConfigProvider` result is a sparse field-level overlay on that baseline. For cache enablement fields, precedence is runtime config, then `defaultConfig`, then DialCache's disabled baseline. For the remote-read deadline, precedence is runtime `remoteReadTimeoutMs`, `defaultConfig.remoteReadTimeoutMs`, `redis.readTimeoutMs`, then the 50 ms library default. - -The disabled baseline sets `requestLocal` to false, leaves the process-local and Redis TTLs unset, and sets `shadow.ramp` to 0% with mismatch logging false. A shared layer with no effective TTL is disabled by policy. When a shared layer has an effective TTL but no effective ramp, its ramp defaults to 100%. Shadow work remains disabled unless `shadow.ramp` is explicitly greater than zero. - -`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined` so the overlay can distinguish omission from an explicit `false`; the effective value still defaults to false after resolution. An omitted `coalesce` is preserved the same way, and its effective value defaults to true, so request coalescing stays on unless a use case explicitly opts out. - -A provider result of `null` (or defensive `undefined`) applies no overrides. An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the baseline. Top-level fields, cache-layer leaves, and leaves inside `shadow` merge independently; an explicit `false` logging flag overrides an inherited `true`. Use explicit values to override inherited policy: `requestLocal: false` disables request-local caching and a layer ramp of `0` disables that shared layer. `DialCacheKeyConfig.disabled()` is the complete new-cache-invocation kill switch in one call: request-local and shadow work off, shadow logging off, and both shared layers ramped to 0. It leaves `coalesce` unset: with every layer off there is no in-flight sharing to disable, and a use case ramped back up at runtime coalesces again unless it explicitly opts out. It does not cancel already-admitted work, and explicit maintenance operations such as `invalidateRemote()` remain available. To stop new cache-invocation Redis reads and fills while preserving other runtime settings, explicitly set both `ramp.remote` and `shadow.ramp` to `0`; the remote ramp alone stops serving but does not override an inherited nonzero shadow ramp. - -DialCache validates `defaultConfig` when `cached()` registers a definition and whenever `getOrLoad()` is invoked: TTLs must be positive safe integers no greater than 31,536,000 seconds (a fixed 365-day duration), remote-read deadlines must be positive safe integers within their documented limit, layer and shadow ramps must be finite percentages from 0 to 100, layer maps and `shadow` must be objects, and `requestLocal`, `coalesce`, and `shadow.logMismatches` must be booleans when present. Invalid defaults are rejected immediately. - -Each registration or one-shot invocation captures an immutable internal snapshot of `defaultConfig`; mutating the supplied config or its maps later does not change that operation's baseline. Runtime policy changes belong in the provider's returned overlay. - -Runtime TTL and ramp leaves are used as supplied instead of falling back to valid default leaves. A TTL outside the same 1-to-31,536,000-second range disables that layer with `invalid_ttl`; a nonnumeric, non-finite, or out-of-range ramp disables it with `invalid_ramp`. Valid ramps include both `0` and `100`. Other layers can still run, and invalid leaves also record a `config_resolution` error so provider garbage is alertable separately from intentional ramp-downs. A malformed runtime config object, layer-map shape, `requestLocal` value, `coalesce` value, explicit `remoteReadTimeoutMs`, or removed top-level `shadowRamp` fails config resolution for the invocation, records `config_error`, and executes the fallback uncached without attempting Redis. The public `DialCacheKeyConfig` constructor and static defaults likewise reject `shadowRamp` immediately; migrate it to `shadow.ramp`. - -An invalid runtime `shadow.ramp` does not affect the cache result or disable an otherwise valid Redis policy. If normal traversal reaches an otherwise shadow-eligible Redis path, DialCache skips shadow work and records a `config_resolution` error. An invalid runtime `shadow.logMismatches` likewise preserves the cache result, Redis policy, shadow result, and shadow metric while suppressing the warning. DialCache validates this diagnostic leaf only after the metrics hook, exact-key cohort, and capacity gates admit shadow work, then records one remote `config_resolution` error for that admitted resolution. - -`cacheConfigProvider` is called for every enabled cache invocation before DialCache performs any cache lookup. Keep it cheap, cache any remote/config-store reads inside the provider, and avoid work that would erase the benefit of a cache hit. +The example below focuses on runtime policy. Remote ramp settings take effect +only when a Redis or Valkey client is configured. ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -const dialcache = new DialCache({ - cacheConfigProvider: async (key) => { - if (key.useCase === "GetUser") { - return new DialCacheKeyConfig({ - // Sparse override: inherit both TTLs and the local ramp from defaultConfig. - ramp: { [CacheLayer.REMOTE]: 25 }, - // Independently sample Redis keys for detached validation/fill. - shadow: { - ramp: 5, - // Emit one warning with a bounded key and native-JSON value strings. - logMismatches: true, - }, - // Can be changed by the provider at runtime for this use case. - remoteReadTimeoutMs: 35, - }); - } - return null; // apply no overrides; use the cached function's baseline - }, -}); - -const getUser = dialcache.cached((userId: string) => db.fetchUser(userId), { - keyType: "user_id", - useCase: "GetUser", - cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ - // Omitted ramps default to 100% because these layers have TTLs. - ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 300 }, - }), -}); -``` - -`ramp` values are percentages from 0 to 100. `0` disables the layer, `100` enables it, and intermediate values are deterministically sampled by cache key and layer, so the same key is consistently sampled in or out of a partial rollout across calls and instances. The assignment algorithm is owned by DialCache and remains stable across releases. Applications that need an externally coordinated cohort can use `cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. DialCache fetches and resolves one config snapshot per enabled invocation. Provider errors do not activate defaults: they fail open, record `config_error`, and execute the fallback function uncached. - -`shadow.ramp` uses the same inclusive 0–100 percentage domain but is independent of cache-layer serving ramps. Omission and `0` disable shadow work; `100` selects every eligible Redis key; intermediate values assign each exact cache key to a stable shadow cohort across calls and instances. A valid remote policy can therefore use `ramp.remote: 0` with a nonzero `shadow.ramp` to exercise and populate Redis without serving from it. A nonzero value explicitly authorizes detached writes after clean shadow-only misses: tracked keys use their watermark-aware write, while untracked keys use their ordinary TTL-based last-writer-wins write. It does not create another `CacheLayer`, activate Redis without a valid remote TTL, or make a request-local/process-local hit continue to Redis. - -Remote serving and shadow sampling use independent deterministic cohorts. Equal partial percentages do not imply the same keys, so a partial shadow cohort does not guarantee that every key admitted by a later partial serving ramp was warmed or validated. Use `shadow: { ramp: 100 }` when every otherwise eligible invocation must exercise the non-serving Redis path before a serving-ramp increase. - -## Cache layers - -### Request-local cache - -Set `requestLocal: true` to memoize resolved values for the lifetime of the outermost `enable()` scope: - -```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; - -const dialcache = new DialCache(); -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetUser", - cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), - }, -); -``` - -`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled `CacheLayer`. The `cacheConfigProvider` can turn it on or off for each invocation. `DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and Redis caching, so request-local caching must be selected explicitly. - -DialCache resolves the runtime config once per enabled invocation and uses it for the entire lookup. When the effective `requestLocal` value is false, the invocation skips request-local lookup and storage without deleting an entry already memoized in the scope. A later invocation that enables request-local caching can reuse that entry. - -The outermost `enable()` call owns the request-local lifetime, and nested `enable()` calls reuse that scope. Request-local state is allocated lazily, only when an invocation enables the layer, so scopes that use only process-local or Redis caching do not allocate it. - -Wrap the complete Node HTTP handler so the request-local scope matches the handler's lifetime: - -```ts -import { createServer } from "node:http"; - -const server = createServer((req, res) => { - void dialcache - .enable(async () => { - const user = await getUser(readUserId(req)); - res.setHeader("content-type", "application/json"); - res.end(JSON.stringify(user)); - }) - .catch((error: unknown) => handleRequestError(error, res)); -}); -``` - -Request-local storage has no capacity limit, eviction, or overflow mode. Entries are retained until the outermost `enable()` callback settles. Use it for short-lived scopes with bounded key cardinality; split long-running streams or large batch jobs into smaller scopes when necessary. - -### Process-local cache - -The process-local layer (`CacheLayer.LOCAL`) uses one LRU per `DialCache` instance. It keeps at most 10,000 entries by default across all use cases while retaining each entry's configured TTL. Set `localMaxSize` to a nonnegative safe integer to change the global entry cap; `0` disables process-local storage: - -```ts -const dialcache = new DialCache({ localMaxSize: 25_000 }); -``` - -The limit counts entries rather than estimating JavaScript object memory. Recently read entries stay resident ahead of less recently used entries when the limit is reached. - -### Redis-backed TTL cache - -The Redis layer supports standalone Redis, Valkey, and Redis Cluster. Register DialCache's bundled node-redis scripts when creating the client, then pass that client to DialCache: - -```ts -import { createClient } from "redis"; -import { DialCache } from "dialcache"; -import { createNodeRedisDialCacheClient, dialcacheRedisScripts } from "dialcache/node-redis"; - -const redisClient = createClient({ - url: process.env.REDIS_URL, - scripts: dialcacheRedisScripts, - disableOfflineQueue: true, - commandsQueueMaxLength: 1_000, - socket: { connectTimeout: 2_000 }, -}); -await redisClient.connect(); - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { - client: createNodeRedisDialCacheClient(redisClient), - // Optional instance default; omit to use DialCache's 50 ms default. - readTimeoutMs: 100, - }, -}); - -async function shutdown(): Promise { - // Stop new work and await every outstanding request-path call and invalidation first. - // Detached shadow work is best-effort and has no drain handle. - await redisClient.quit(); -} -``` - -`redis.client` is required when Redis is configured and accepts the semantic `DialCacheRedisClient` interface. `redis.readTimeoutMs` is optional and sets the instance default for remote reads; omit it to use 50 ms. Create and connect the underlying client before constructing `DialCache`. Node-redis users should register the supplied mutation scripts and wrap their client with `createNodeRedisDialCacheClient` as shown above; the adapter performs reads with native commands. The registered `dialcache*` methods are DialCache's wiring, not a write API: they return raw script replies — the stamp's `2` means the placeholder was lost, not success — so code invoking them directly must map stamp replies through `resolveTrackedRedisWriteReply` from `dialcache/redis-protocol`. The helper requires node-redis's promise API and does not support `legacyMode`, whose callback surface and `.v4` view do not expose the complete native-command-plus-custom-script contract together. +const runtimePolicies = new Map(); -Valkey GLIDE users pass an already-created standalone or cluster client and its -module namespace to the GLIDE adapter: - -```ts -import * as valkeyGlide from "@valkey/valkey-glide"; -import { DialCache } from "dialcache"; -import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; - -const glideClient = await valkeyGlide.GlideClient.createClient({ - addresses: [{ host: "127.0.0.1", port: 6379 }], - requestTimeout: 2_000, - advancedConfiguration: { connectionTimeout: 2_000 }, -}); -const redisClient = createValkeyGlideDialCacheClient(glideClient, valkeyGlide); const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, -}); - -function shutdown(): void { - // After draining request-path calls and invalidations, close GLIDE; the - // adapter is stateless. Detached shadow work is best-effort and has no - // drain handle. - glideClient.close(); -} -``` - -Pass the same GLIDE 2.x module namespace that created the client. The adapter -uses that namespace's `GlideClient` and `GlideClusterClient` identities, -`Batch` and `ClusterBatch` constructors, and `Decoder.Bytes` without importing -a GLIDE runtime itself. The helper accepts a direct official client instance and -fails during construction when the client came from another module instance or -is hidden behind a forwarding wrapper, because it cannot safely infer that -wrapper's topology. Custom wrappers can implement `DialCacheRedisClient` -directly. - -The application owns the complete Redis lifecycle. It creates and connects the underlying client and passes the semantic adapter to DialCache. During shutdown, stop starting DialCache-backed work and await every promise returned by a cached function, `getOrLoad()`, or `invalidateRemote()`, including calls still running fallbacks that may later write Redis. A read that crossed DialCache's wait deadline may still be active inside the client, so use client-native telemetry and shutdown controls to drain or terminate that work before closing the connection. DialCache only borrows `redis.client`; it has no close or drain method and never disposes or closes caller resources. - -Awaiting those public promises does not drain detached shadow work. Shadow scheduling and deadline timers are unreferenced and completion is not guaranteed during shutdown; Redis operations, source reads, serializers, and asynchronous telemetry already started by shadow work remain caller-owned and may still be active. Stop new work before closing their dependencies and accept that an in-flight shadow fill may have been dispatched even if its final outcome is lost during teardown. DialCache does not add a shutdown hook or keep the process alive to deliver best-effort outcomes. - -Neither adapter owns additional resources: both dispatch their mutation scripts by source SHA1 and hold no native handles, so the application simply closes the underlying client after draining work. Applications that construct their own GLIDE `Script` objects should know that on GLIDE 2.0.0, releasing a handle has been observed to break other live handles for the same script source despite GLIDE's documented reference counting. - -Reads use native `GET` for untracked entries and one atomic `MGET` for each tracked value-and-watermark pair. The adapters validate and decode the returned frame in the Node process. Tracked reads are deliberately routed to primaries so a lagging replica cannot hide an invalidation watermark. - -Writes are native too, so the payload never crosses the Redis-to-Lua boundary. An untracked write is one `SET` of a client-encoded frame. A tracked write pipelines two ordered commands on one connection: a `SET` of a version-0 placeholder frame carrying a fresh per-write nonce, then the small payload-free `WRITE_TRACKED_STAMP_SCRIPT`, which fences against the watermark, promotes exactly the placeholder carrying its nonce to a served frame with Redis server time, and maintains the watermark TTL. A placeholder is unreadable on both read paths until promoted, so an interleaved or lost stamp degrades to a miss that expires with the value TTL rather than partial state — including briefly blanking a previously readable key the write replaces. The nonce means the stamp can never revive a frame it does not own: if its paired `SET` was rejected, overwritten, or expired, the stamp reports the placeholder gone and the write fails with `DialCacheRedisPlaceholderLostError` as a `cache_write` error instead of publishing another write's leftovers. Losing a same-key write race is one such outcome, so `cache_write` carries a benign, self-healing floor that concentrates on hot tracked keys at TTL expiry — size write-error alerts for it. The `cache_write` metric itself stays one bounded counter; the error's class and name distinguish the lost-placeholder case in logs, and in the `catch` blocks of code that calls an adapter's `write()` directly — DialCache's own request paths absorb it fail-open rather than rethrowing to callers. Each occurrence also emits one warn through the configured logger (the default is `console`), so fleets expecting hot-key write contention should supply a logger that rate-limits or filters that class. A `SET` failure is the write's outcome even when the stamp settled. The pair is deliberately not a `MULTI`/`EXEC` transaction, which would consume caller-owned `WATCH` state. - -Native commands retain Redis's wrong-type behavior. An untracked `GET` surfaces `WRONGTYPE`; tracked `MGET` represents a wrong-type member as a missing value. A wrong-type tracked value is therefore a clean miss and may be replaced with a valid DialCache frame after the fallback succeeds, while a wrong-type watermark prevents the tracked write from succeeding: its paired `SET` still lands, leaving only an unreadable placeholder until expiry or a later successful write. - -Node-redis forces tracked cluster commands to the slot primary. GLIDE uses an explicit primary route in cluster mode; in standalone mode it sends `MGET` through a one-command non-atomic batch because direct read commands follow the client's replica-read preference. Standalone batches use the primary, and `MGET` itself provides the atomic snapshot without consuming caller-owned `WATCH` state. The GLIDE helper distinguishes those modes from the direct client's runtime identity and rejects ambiguous clients instead of silently choosing a route. - -For the stamp and invalidation scripts, node-redis computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` after `NOSCRIPT`; its cluster client routes commands by their first key and performs that fallback on the selected shard. That retry likewise extends the unreadable-placeholder gap of a tracked write by one round trip on a cold script cache. The GLIDE adapter batches the tracked write's `SET` with an `EVALSHA` of the stamp script — routing cluster write batches to the slot primary — and recovers from a flushed script cache by re-sending the stamp as `EVAL` with its source, which the server caches under the same SHA1, so the first tracked write against a cold script cache pays one extra round trip. A late stamp stays paired to its own placeholder through the nonce; if the placeholder is gone by then, the write fails rather than publishing. Invalidation dispatches as `EVALSHA` by the script's source SHA1 on both adapters, and both retry a rejected dispatch once by re-sending the source as `EVAL`: the invalidation script is idempotent — its watermark only advances and its TTL only widens — so a duplicate execution after an ambiguous failure is harmless, and the retry heals a flushed script cache and an `EVALSHA`-rejecting proxy without depending on error wording. Reply-domain violations are deterministic and are not retried. When the retry also fails, the surfaced error is the retry's. On GLIDE the original rejection is attached as the retry error's `cause` unless it already carries one, and a failing invalidation is bounded by roughly two `requestTimeout` windows. On node-redis the retry rejection surfaces unmodified and the original is discarded — the library rejects every command flushed by a single disconnect with one shared error instance, so the adapter never mutates it — and no per-command deadline exists: `disableOfflineQueue`, `commandsQueueMaxLength`, and `reconnectStrategy` bound queueing and dispatch (the setup snippet above disables the offline queue, which makes a disconnected retry fail fast instead of waiting for reconnect), but a command already written to a hung connection has no reply deadline. Its own `NOSCRIPT` recovery may also add one round trip before the adapter's retry. A retry that heals is indistinguishable from a first-attempt success in DialCache's metrics and logs. The genuinely silent regime is invalidation-dispatch healing: watch server-side `INFO commandstats` for `cmdstat_eval` calls rising in step with invalidation volume while `cmdstat_evalsha` stays flat (a proxy rejecting `EVALSHA` before it reaches Redis) or accrues `rejected_calls` (an ACL denial). A sustained stamp fault is loud by contrast — the ACL paragraph below describes its amplitude. - -A tracked write rejected by an active future watermark uses `UNLINK` to remove the value key — the placeholder it just stored, along with any logically stale frame — without synchronously freeing it on Redis's command path. The mutation protocol therefore requires a server that implements `UNLINK` (Redis 4.0 or later, or a compatible Valkey release). Command-restricted Redis ACLs must allow the client to issue the native `GET`, `MGET`, and `SET` commands — `SET` newly carries every write, where the previous protocol wrote only through scripts — plus `EVALSHA` (the steady-state dispatch for both mutation scripts) and `EVAL` (both adapters recover a flushed script cache by re-sending script sources, never via `SCRIPT LOAD`). Server versions differ on whether script-invoked commands are also checked against the invoking user, so grant what the mutation scripts invoke as well: `TIME`, `GET`, `SET`, and `PTTL` (both scripts), plus the stamp's `PEXPIRE`, `UNLINK`, `GETRANGE`, and `SETRANGE`. Verify those grants before upgrading, because the failure amplitude of a persistent stamp fault changed. A sustained stamp failure (denied command, a proxy rejecting `EVALSHA`) still lands every paired `SET`, so each tracked write replaces the last served value with an unreadable placeholder while also suppressing process-local publication — within one TTL horizon the source absorbs full traffic, where the previous protocol degraded to serving stale values until expiry. DialCache's integration matrix covers Redis 6.2 and Valkey 8. - -#### Remote read deadlines and async liveness - -DialCache bounds every active Redis read. The effective timeout is resolved per use case and per invocation: runtime `remoteReadTimeoutMs`, then `defaultConfig.remoteReadTimeoutMs`, then optional instance `redis.readTimeoutMs`, then 50 ms. Values must be positive safe integers no greater than 2,147,483,647. There is no unbounded escape hatch for remote reads. - -When the deadline expires, DialCache aborts the optional `RedisReadContext.signal`, records one `cache_read_timeout` error, logs a `RedisReadTimeoutError`, and starts the source fallback. Late read fulfillment or rejection is consumed and ignored. A read failure or timeout never triggers a post-fallback Redis write; an untracked active process-local miss may retain the source value, while a tracked key suppresses local publication because the failed read did not establish watermark safety. - -Same-key followers share the leader's remaining remote-read budget. The timer covers only the semantic Redis read, not config resolution, serializer load, fallback work, Redis writes, or invalidation. `fallbackTimeoutMs` starts separately when the source fallback begins. - -The bundled node-redis adapter passes the signal through per-command options, which can remove queued work where supported. Aborting after dispatch does not unsend a command or prove that Redis stopped executing it. GLIDE's current adapter commands have no per-invocation signal, so a read may continue after DialCache has fallen back. Keep client-native connection, retry, queue, and response budgets in place; they bound underlying resource lifetime while DialCache's deadline bounds caller wait time. - -Writes, invalidations, async `cacheConfigProvider` calls, and custom serializer methods still need finite application-owned budgets. Client support differs: GLIDE's `requestTimeout` bounds every command's reply wait, while node-redis has no per-command deadline — its queue and reconnect controls bound admission and dispatch only (see the invalidation-retry paragraph above), so bound node-redis mutations at the connection layer rather than per call. Do not put mutations behind a bare `Promise.race`: rejecting the outer promise neither removes queued work nor proves whether a dispatched mutation executed. - -#### Serialization - -The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not expose client commands or wire encodings. The shared `encodeRedisFrame`, `encodeTrackedRedisPlaceholder`, `decodeRedisFrame`, and `decodeTrackedRedisFrame` helpers, the `resolveTrackedRedisWriteReply`, `validateRedisSetReply`, and `validateRedisScriptInvalidationReply` reply helpers, the `ceilSupportedCacheTtlMs` TTL guard, and the tracked stamp and invalidation Lua sources are available from `dialcache/redis-protocol`, so custom adapters can reuse the bundled adapters' exact framing, miss, watermark-fencing, TTL-domain, and reply rules. A custom tracked write must pass the stamp script `KEYS = [valueKey, watermarkKey]` and `ARGV = [cacheTtlMs, nonce]`, running `cacheTtlMs` through `ceilSupportedCacheTtlMs` and using the result for both the paired `SET`'s `PX` and `ARGV[1]` (the stamp script re-validates the same domain server-side as defense in depth), with the nonce from the same `encodeTrackedRedisPlaceholder` call; `resolveTrackedRedisWriteReply` maps the reply, failing the write with the root-exported `DialCacheRedisPlaceholderLostError` when the stamp replies `2`. Custom adapters can throw the root-exported `DialCacheRedisPayloadError`, `DialCacheRedisPayloadEncodingError`, `DialCacheRedisProtocolError`, and `DialCacheRedisPlaceholderLostError` classes to distinguish malformed replies, unsupported encodings, reply-domain violations, and lost placeholders in logs. DialCache records bounded `cache_read`, `cache_write`, or `invalidation` metrics by failure site. - -Redis values use a compact binary frame: - -```text -byte 1 format version: 1 = servable, 0 = unreadable tracked placeholder -bytes 2-9 uint64 big-endian: Redis server time for a promoted tracked frame, - informational client time for an untracked frame, or the random - per-write nonce while a tracked placeholder awaits its stamp -byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload (optionally zstd-compressed; see Compression) -``` - -Adapters build frames in the Node process. Untracked frames come from `encodeRedisFrame` and carry an informational client-clock timestamp that untracked reads never consult. Tracked frames start as `encodeTrackedRedisPlaceholder` output — version byte `0`, with a random per-write nonce in the timestamp bytes — which no read path serves; the stamp script verifies the nonce and promotes the frame to version `1` with Redis server time using Lua's `struct` library, and adapters decode it with Node's buffer primitives. Redis TTL is authoritative, so expiry metadata is not duplicated in the frame. `payload` is produced by the operation's serializer, or by `JsonSerializer` by default. Custom serializers can return either `string` or `Buffer`. Payloads stored raw keep their exact serialized bytes: strings are stored as UTF-8 and Buffers byte-for-byte without base64 expansion, except that binary output beginning with a [compression envelope byte](#compression) (`0x00`–`0x02`) gains a one-byte escape prefix on the wire. Payloads at or above the compression threshold may instead be stored as a zstd envelope (see [Compression](#compression)), so wire bytes for large values are not the serializer's output. Adapters return the frame payload as-is; the envelope — including restoring a compressed string's representation before `serializer.load` — is interpreted by the core above them. - -DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no runtime validation pass, so the default adds no traversal beyond JSON serialization itself. A top-level `undefined` result is supported with an internal sentinel. - -When `serializer.load` rejects a Redis payload, DialCache records a `serialization_load` error, counts the read as a remote cache miss, runs the fallback, and attempts to replace the rejected payload. A validating custom serializer can therefore treat an incompatible cached value as a refreshable miss without adding a schema version to the cache key. - -`JsonSerializer` validates JSON syntax only. It cannot detect that a structurally valid payload came from an incompatible application value schema. Applications that keep the same `useCase` across deployments must keep default-JSON values backward compatible. For an incompatible change, either provide a serializer whose `load` method validates and rejects the old shape, or change `useCase` to isolate the new cache entries. On the caller-serving Redis path, mutually incompatible validating serializers in a mixed deployment can repeatedly reject and replace each other's values; correctness is preserved, but expect additional fallback and Redis-write load until the rollout converges. Shadow work reports a non-null payload that fails `load` as `deserialization_error` and never replaces it. - -When a cached function or inline loader's resolved return type is statically JSON-compatible, `serializer` remains optional. This includes JSON primitives, arrays, plain object/interface shapes, optional object fields, and a top-level `undefined`. Types known not to survive the default round trip require a typed `Serializer`: - -```ts -import { DialCache, type Serializer } from "dialcache"; - -const dialcache = new DialCache(); -const dateSerializer: Serializer = { - dump: (value) => value.toISOString(), - load: (value) => new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), -}; - -const getUpdatedAt = dialcache.cached( - (userId: string) => db.fetchUpdatedAt(userId), - { - keyType: "user_id", - useCase: "GetUpdatedAt", - cacheKey: (userId) => userId, - serializer: dateSerializer, - }, -); -``` - -The compile-time guard rejects known incompatible shapes such as `Date`, `Map`, `Set`, `bigint`, symbols, functions, Buffers, typed arrays, method-bearing class instances, required nested `undefined`, `unknown`, and `any`. It applies to every `cached()` declaration and `getOrLoad()` invocation because active layers are selected at runtime. A global Redis serializer is not parameterized by each returned type, so it cannot discharge this requirement; non-JSON operations must select a typed serializer. - -This guard is deliberately conservative and is not a proof of runtime data. TypeScript cannot detect non-finite numbers, cyclic/shared references, runtime getter or `toJSON` behavior, or data-only class instances that look like plain objects. Opaque, generic, or deeply recursive types may also require an explicit serializer. Providing `Serializer` (including an explicitly typed `JsonSerializer`) is a trusted caller assertion; DialCache does not serialize-and-deserialize again to validate it. - -#### Compression - -DialCache transparently compresses serialized Redis payloads with zstd (level 3, via `node:zlib`) when they are at least 4096 serialized bytes, and stores the compressed form only when it is smaller than the raw stored form. Compression sits below the serializer and above the Redis client, so serializers, adapters, and the frame layout are unaffected. The first byte of a binary frame payload written by a release with payload compression is an envelope byte: `0x01` marks a compressed UTF-8 string and `0x02` compressed binary output, each followed by the zstd frame, while `0x00` is an escape prefix for raw binary serializer output whose own first byte is `0x00`–`0x02` (readers strip the prefix and never decompress it; the escape applies even with `compression: false`). Payloads below the threshold are otherwise stored byte-identical to earlier DialCache releases; only binary output beginning with an envelope byte gains the one-byte escape. - -Decompressed payloads are capped at 512 MiB, mirrored on the write side by refusing to compress anything larger, so no writable entry is unreadable and a corrupt or hostile entry cannot force a giant synchronous allocation. zstd runs synchronously on the event loop: at level 3 it stays cheaper than the adjacent `JSON.stringify`/`parse` at every size (~2 ms to compress 2 MiB), but cost rises steeply with level — measured ~250 ms for 1 MiB at level 19 and ~1.5 s for 2 MiB at level 22 — so treat high levels as an informed opt-in and watch the compression timer metric. - -Compression is on by default and configured per instance next to the serializer: - -```ts -const dialcache = new DialCache({ - redis: { - client: redisClient, - // Defaults shown; pass compression: false to store every payload uncompressed. - compression: { thresholdBytes: 4096, level: 3 }, - }, -}); -``` - -Reads always decompress marked payloads regardless of this setting, so disabling compression never orphans previously written entries. The escape prefix makes decoding exact for every entry written by a release with the envelope, whatever bytes a custom serializer emits. Entries written by older releases have no envelope, which leaves a bounded residual until they expire: a legacy binary payload beginning `0x01`/`0x02` is handed back untouched when zstd rejects it (`fallback_raw`), but one whose remaining bytes parse as a zstd stream is misread, and a legacy payload whose first two bytes are both in `0x00`–`0x02` loses its first byte to the escape strip. If a custom binary serializer can emit such output, bump its use case or key type when upgrading so old entries are simply misses. - -Rolling upgrades and rollbacks degrade to misses, not errors, but are visible in metrics: during a mixed-fleet window, readers on releases without payload compression fail `serializer.load` on compressed entries, producing a transient `serialization_load` error spike (and shadow `deserialization_error` outcomes) plus refill churn until the fleet converges — expected noise, worth an alerting note. For a zero-noise upgrade with string/JSON serializers, deploy this release with `compression: false` first, then enable it once the fleet converges; binary serializers with envelope-colliding output still write escaped bytes in phase one, so rely on key versioning there instead. Rolling back to a release without the envelope degrades compressed and escaped entries to refreshable misses the same way, presuming `serializer.load` rejects the foreign bytes — a permissive binary decoder could misread an escaped payload instead, the same caveat as the forward residuals above. On runtimes without `node:zlib` zstd (Node below 22.15, and 23.0–23.7), ESM consumers cannot load the package at all (the import fails), while CommonJS consumers fail at construction when compression is enabled; `compression: false` is the working configuration there for CommonJS only. - -Each write records a bounded compression outcome (`compressed`, `below_threshold`, `not_smaller`, or `write_over_limit`) and, when compressed, a compressed-to-original size ratio; reads record `decompressed`, `fallback_raw`, or `read_over_limit` (`write_over_limit` is a capacity signal; `read_over_limit` a corruption/integrity signal). Compression and decompression latency is observed separately with an `operation` label (see [Metrics](#metrics)). Payload sizes are reported at both stages: the size histogram observes serializer output (pre-compression, the distribution to consult when tuning `thresholdBytes`), and the stored-size histogram observes what was actually written after compression and escaping — the difference between their sums is the bytes compression saved. - -#### Shadow validation - -Shadow mode runs a sampled, detached Redis path for tracked or untracked keys without allowing that path to serve the caller. A Redis hit is compared with the source of truth (SoT); a clean Redis miss can be filled from the caller-accepted SoT value. Redis serving and shadow execution are independent, and shadowing is opt-in per use case through `shadow.ramp`: - -```ts -import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, - // A bundled or custom adapter with shadowValidation support is required. - metrics, - // At most one scheduled or running shadow job by default; tune deliberately. - shadowMaxInFlight: 4, + cacheConfigProvider: (key) => runtimePolicies.get(key.useCase) ?? null, }); const getUser = dialcache.cached( @@ -521,403 +175,339 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - // Optional for shadowing; adds watermark fencing to Redis reads and fills. - trackForInvalidation: true, - // Optional: override strict deep equality with use-case semantics. - shadowComparator: (cached, source) => - cached.id === source.id && cached.version === source.version, defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.REMOTE]: 300 }, - // Exercise and populate Redis without serving it to callers. - ramp: { [CacheLayer.REMOTE]: 0 }, - shadow: { - ramp: 5, - // Default-off warning with a bounded key and native-JSON value strings. - // Enable only after approving the logger and data-handling policy. - logMismatches: true, + ttlSec: { + [CacheLayer.LOCAL]: 60, + [CacheLayer.REMOTE]: 60, + }, + ramp: { + [CacheLayer.LOCAL]: 0, + [CacheLayer.REMOTE]: 0, }, + shadow: { ramp: 0 }, }), }, ); -``` - -Shadow work is eligible only when a valid remote TTL/policy exists, its effective `shadow.ramp` selects the exact cache key, a configured metrics adapter implements `shadowValidation`, and capacity is available. Tracked and untracked Redis keys are both eligible; each keeps its existing read and write mode. Logging is supplemental to the metric; enabling `logMismatches` does not activate shadow work without the metrics hook. The bundled Prometheus and Datadog adapters implement that hook. There are two paths: - -- When remote serving is enabled and produces a Redis hit, DialCache retains the exact serialized payload that supplied the caller as `C0`. -- When the remote policy is valid but disabled specifically by `ramped_down`, DialCache starts a detached Redis read for `C0` using the key's existing tracked or untracked mode. Its result can be validated or used to decide whether a clean miss may be filled, but can never supply the caller or populate an in-memory layer. - -A missing or invalid remote policy, config-provider failure, absent Redis client, disabled call, omitted metrics hook, zero/omitted shadow ramp, cohort exclusion, capacity rejection, or earlier request-local/process-local hit does not launch a shadow-only Redis path. Shadow work begins only if normal traversal reaches the Redis layer. A normally enabled remote miss already follows the caller's ordinary fallback-and-fill path and does not launch a duplicate shadow fill. - -On a served hit, DialCache returns the already-decoded cached value before starting the SoT read or any confirmation work. On a ramped-down path, the caller invokes and awaits its normal configured fallback exactly once and receives only that result; detached work reuses the same accepted `S` instead of calling the loader again. The caller never awaits shadow `C0`, comparison, confirmation `C1`, shadow serialization, or fill. Slow, failed, or timed-out shadow work cannot delay, reject, or change the caller result. - -The detached job uses this bounded algorithm: - -1. Obtain the original Redis payload as `C0` using the key's existing tracked or untracked read mode. -2. If `C0` is missing, wait for the caller's successfully accepted `S` after its configured fallback boundary and attempt one normal Redis write in the same mode using the resolved TTL. Before the whole-job deadline, emit `filled` when Redis accepts it, `fill_blocked` when a tracked invalidation watermark rejects it, or `fill_error` when serialization or the write fails. `fill_blocked` is not produced by compliant untracked writes. -3. If `C0` is non-null, obtain `S`, deserialize an isolated snapshot of `C0`, and run the default or custom semantic comparator. Any non-null `C0` is observation-only: DialCache never repairs or overwrites it, including when deserialization fails. -4. If `C0` and `S` match semantically, emit `match` without another Redis read. -5. Otherwise, reread Redis directly in the same mode as `C1`, bypassing request-local and process-local cache. -6. If `C1` is missing or differs byte-for-byte from `C0`, emit `superseded`; if it is identical, emit `mismatch`. -7. If the confirmation read fails or reaches its Redis-read deadline, emit `confirmation_error`. - -Here a clean miss means the semantic Redis read returned `null`; it does not include a non-null payload that later fails deserialization. A caller fallback rejection or timeout never becomes accepted `S` and never starts the fill. - -Both detached Redis reads use the effective `remoteReadTimeoutMs` and the key's normal protocol. Every clean-miss fill uses the same serializer, TTL, and timestamp semantics as an ordinary fill — server time for tracked fills, informational client time for untracked ones. A tracked fill blanks the key with its placeholder before publishing, so a lost or raced stamp can leave a previously readable value unreadable until the value TTL, and `fill_error` includes that benign lost-placeholder outcome. Tracked `C0` and `C1` reads remain watermark-aware and are routed to primaries by the bundled adapters, while tracked fills also retain the ordinary invalidation watermark. Untracked reads use the ordinary one-key read route, which has no shadow-specific primary guarantee, and untracked fills use the ordinary TTL write without a watermark. Strings compare exactly, Buffers compare by bytes, and string/Buffer pairs compare by their UTF-8 bytes. DialCache does not deserialize `C1`, compare it with `S`, or chase another version. - -The detached scheduler, Redis-read deadline timers, and overall shadow deadline timer are unreferenced, so they do not keep an otherwise idle process alive. Detachment is asynchronous work on the Node event loop, not a worker thread: synchronous source, serializer, or comparator work can still occupy the event loop after the request path has been released. - -Detached execution retains the original `cached()` argument references or `getOrLoad()` loader closure; DialCache cannot generically clone them. Treat object arguments, captured source-selection state, and the returned `S` as immutable, or snapshot them before invoking DialCache. Mutating them after the caller continues can compare or serialize a value that no longer corresponds to the already-built key. - -`shadowMaxInFlight` is a per-instance positive safe integer and defaults to `1`. It counts admitted jobs until shadow-owned Redis/source/serializer/comparator work settles, including detached reads or dispatched writes whose DialCache deadline already elapsed. The optional `C1` and clean-miss fill remain in the original slot. On a ramped-down path, shadow work shares the caller's SoT promise; once the shadow deadline expires, the raw caller-owned loader may continue without retaining the shadow slot, including when `fallbackTimeoutMs` is `null`. DialCache also suppresses another job for the same exact key while shadow-owned work remains active. There is no queue: exact-key duplicates and work above the instance cap are dropped and reported as `dropped`. Separate instances have independent limits, so this is not a fleet-wide source-of-truth or Redis concurrency cap. - -Each job has one monotonic deadline across detached `C0`, the SoT result, serializer work, comparison, optional `C1`, and clean-miss fill. Served-hit timing begins when its detached validation callback starts. On a ramped-down path, timing begins immediately before the caller's SoT invocation so synchronous source work that runs before admission still consumes the same budget. Each Redis read also has its effective read deadline. A finite `fallbackTimeoutMs` is reused as the overall shadow budget. When `fallbackTimeoutMs` is `null`, the normal fallback remains intentionally unbounded, but detached shadow work still uses a 60-second budget. Once timeout delivery marks a job abandoned, DialCache releases retained `C0` references and prevents later phases from starting. - -JavaScript promises and Redis writes do not provide a general cancellation or transaction boundary. Work already dispatched may continue and keeps the shadow slot until it settles. A write rejection, `fill_error`, or shadow `timeout` after dispatch does not prove that Redis was unchanged; the command may have executed before its result became unavailable. Conversely, `filled` means the semantic client returned success before the shadow deadline, not that the value is still present. Give dependencies finite native budgets and treat shadow outcomes as best-effort operational evidence. - -A `match` means application-level value equality: - -- By default, DialCache uses Node's `util.isDeepStrictEqual`. Plain-object property insertion order does not affect the result; values, array order, prototypes, constructors, Buffers, Maps, Sets, and other supported structures remain strictly compared. -- An optional typed `shadowComparator(cachedValue, sourceValue)` on `cached()` or `getOrLoad()` can define narrower domain equality, such as ignoring a volatile timestamp. It must synchronously return a boolean and must be deterministic, side-effect-free, non-mutating, and bounded. -- A comparator throw or non-boolean return is `comparison_error`, never `mismatch`. An accidental promise is not accepted as a comparison result; DialCache consumes its settlement while retaining the shadow slot, subject to the same detached deadline. - -DialCache retains the semantic `string | Buffer` returned by the Redis client but never exposes it to the comparator. After the source read completes, detached work calls the same effective serializer's `load` method to create an independent cached snapshot, then compares that snapshot with the raw value returned by the source loader. It does not reuse the cached object already returned to a served-hit caller, so caller mutation cannot contaminate validation. No payload copy, shadow deserialization, deep comparison, or hash is added to the served-hit request path. -The effective serializer's `load` method therefore runs a second time for a sampled served hit and once in detached work for a shadow-only hit. It must be repeatable, non-mutating, and return independently usable values. On a clean miss, its `dump` method may run after the caller has received `S`, so `S` must remain immutable through detached serialization. A custom `DialCacheRedisClient` must return an operation-owned payload whose string/Buffer contents remain stable after `read()` settles. Comparing the deserialized cached snapshot with the raw source value intentionally detects lossy serialization; use a custom comparator only when such normalization or ignored fields are valid use-case semantics. - -Shadow metrics use the bounded outcomes `match`, `mismatch`, `superseded`, `filled`, `fill_blocked`, `fill_error`, `redis_error`, `source_error`, `deserialization_error`, `comparison_error`, `confirmation_error`, `timeout`, and `dropped`. `redis_error` applies to the initial shadow-only `C0` read; `confirmation_error` applies to `C1`; `fill_blocked` applies only when a tracked watermark rejects the write. A clean `C0` miss is an ordinary `miss{layer="remote_shadow"}` and terminates with a fill, source, or timeout outcome rather than a second shadow outcome for the miss itself. Labels never contain cache ids, values, payloads, Redis keys, or raw exception text. - -Confirmed-mismatch logging is separately opt-in through `shadow.logMismatches`; it never replaces the required `shadowValidation` metric or emits for a mismatch candidate that becomes `superseded`. The single warning contains `cacheNamespace`, `useCase`, `keyType`, `outcome: "mismatch"`, `cacheKey`, `cachedValueJson`, and `sourceValueJson`. `cacheKey` is the logical DialCache URN capped at 2 KiB, not the physical Redis storage key. DialCache independently applies native `JSON.stringify` to the deserialized cached snapshot and raw source value supplied to the comparator, then caps each resulting string at 8 KiB. A byte-clipped field ends in `...[truncated]`, counted inside its cap. If native JSON throws or returns `undefined`, the corresponding JSON field is `null`; the other side is still attempted. DialCache does not compute a textual diff or call the configured serializer again for logging. - -Mismatch logging is intentionally default-off. Logical URNs can contain ids and arguments, while cached and source values may contain secrets or personal data; truncation is not redaction. DialCache creates the JSON strings only after terminal mismatch confirmation and never passes the raw compared-value references to the logger. Native JSON semantics apply: getters and `toJSON` methods may run, unsupported values may be omitted or normalized, and cycles or `bigint` can make a field unavailable. Stringification is synchronous, and the 8 KiB caps apply only after `JSON.stringify` returns; they do not bound input traversal, hook execution, event-loop time, or the intermediate JSON string. Enable mismatch logging only for trusted, reasonably bounded values and with an approved logger, redaction, transport, access, and retention policy. - -The byte caps apply before logger framing or escaping, so they do not guarantee a final transport event below a sink-specific limit; the metadata fields are not size-clamped. A detail-construction failure degrades to the metadata-only warning. Logger throws and rejected promises or thenables remain isolated from cache and shadow correctness. - -Detached Redis reads, serializer loads/dumps, payload sizes, and read/write errors use the existing layer label with `layer="remote_shadow"`. This distinguishes non-serving Redis cost from caller-path `layer="remote"` telemetry without adding a metric or label. The established `observeGet{layer="remote"}` boundary includes caller-path deserialization, while `observeGet{layer="remote_shadow"}` ends when the deadline-bounded Redis read result settles; detached serializer work and any later raw-client settlement are outside that timer. The request-path read that supplied a served `C0` keeps `layer="remote"`, and a ramped-down caller keeps `disabled{layer="remote", reason="ramped_down"}`. No `disabled{layer="remote_shadow"}` event is emitted for ineligible or dropped work; `dropped` remains the terminal shadow outcome. Confirmation reads use the same `remote_shadow` value, with `superseded` or `confirmation_error` describing their role. - -The command amplification is bounded: a selected served hit adds one SoT read and adds `C1` only for a semantic mismatch candidate; a selected ramped-down hit adds detached `C0`, reuses the caller's existing SoT read, and likewise adds `C1` only for a candidate; a selected ramped-down miss adds detached `C0` and at most one fill in the key's existing mode. `superseded` means only that the original observation could not be confirmed. `mismatch` means the exact `C0` payload survived another Redis read after the SoT disagreement; it is not a cross-system atomic snapshot or a guarantee that the mismatch persists. For an untracked key it is also not proof of primary freshness or invalidation safety. - -The initial `C0` read and later fill are not atomic. The fill is a normal overwrite, not a compare-and-set or write-if-still-missing operation: another writer can populate Redis after the clean miss and be overwritten by the shadow fill. Tracked invalidation watermarks still fence tracked writes using Redis time, so size `futureBufferMs` to cover the complete SoT, serialization, client queue, network, and write interval when stale-publication protection matters. An untracked shadow fill has no such fence and retains the ordinary TTL-based last-writer-wins contract; because it is detached, an older accepted source value may be written after a concurrent source mutation and remain until expiry. Shadow mode never repairs a non-null `C0`, refreshes its TTL, invalidates, evicts local state, or changes the value returned to the caller. - -A served-hit sample invokes the wrapped function or inline loader as an additional source read, so that loader must be safe to call for observation. A ramped-down sample reuses the caller's ordinary invocation and does not add another SoT call. - -For valid policies, shadow-specific source calls, cache-path Redis traffic, returned values, and metrics are unchanged when `shadow` is omitted or `shadow.ramp` is `0`. Shadow policy is grouped under `DialCacheKeyConfig.shadow`; consumers of the former flat ramp field must migrate to `shadow: { ramp }`. The public constructor, static defaults, and runtime provider results reject the removed field. `DialCacheKeyConfig.disabled()` explicitly disables the shadow ramp and mismatch logging. `shadowComparator` remains a typed `cached()` / `getOrLoad()` option because it defines stable use-case equality, while `shadowMaxInFlight` remains a per-instance concurrency limit. The clean-miss bootstrap adds no additional ramp knob, Redis protocol operation, metric instrument, or label key; enabling shadowing authorizes the same-mode Redis write described above. Untracked keys now participate when they have a nonzero effective shadow ramp and an observable metrics hook, so deployments that previously supplied such a ramp while relying on the tracked-only eligibility rule must set it to `0` before upgrading if they do not want the added SoT reads, Redis traffic, possible fills, and opted-in mismatch logs. Exported unions include `remote_shadow` in `MetricLayer` and `superseded`, `filled`, `fill_blocked`, `fill_error`, `redis_error`, and `confirmation_error` in `ShadowValidationOutcome`. TypeScript consumers with exhaustive switches or `Record` values must include those cases, and dashboards restricted to `layer="remote"` intentionally exclude detached traffic. - -## Cached-value ownership - -Treat values returned by cached functions or `getOrLoad()` as immutable. DialCache does not clone or freeze values stored in request-local or process-local memory. Mutating a cached object can therefore be observed by later callers in the same request, callers in other requests that hit the process-local cache, or callers that coalesced onto the same in-flight result. - -This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed arrays, and class instances. Redis deserialization can produce a different reference from an in-memory hit, so reference identity is layer-dependent and is not part of the API contract; never rely on a specific layer cloning a value before mutation. - -If a caller needs a mutable value, copy it explicitly before changing it: - -```ts -const sharedUser = await getUser("123"); -const editableUser = structuredClone(sharedUser); -editableUser.displayName = "New name"; -``` - -Use a narrower copy when its semantics are sufficient; the ownership boundary is the caller's responsibility. - -## Targeted invalidation and watermarks - -Mutable Redis-backed use cases can opt into targeted invalidation by setting `trackForInvalidation: true` in the options and calling `dialcache.invalidateRemote(keyType, id, futureBufferMs)` after writes. `invalidateRemote` requires `DialCacheConfig.redis`; local-only caching remains supported, but this explicit remote maintenance operation rejects when Redis is absent. The buffer is an application-owned safety value; DialCache cannot choose a universally safe nonzero value: - -```ts -import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -import { createNodeRedisDialCacheClient } from "dialcache/node-redis"; - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: createNodeRedisDialCacheClient(redisClient) }, -}); - -// Chosen from this application's clock-skew bound and measured worst-case source/fallback timings. -const USER_INVALIDATION_BUFFER_MS = 5_000; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetMutableUser", - cacheKey: (userId) => userId, - trackForInvalidation: true, - // Strongly invalidated mutable data should disable request-local and process-local caching. - defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.REMOTE]: 300 }, - ramp: { [CacheLayer.REMOTE]: 100 }, - }), - }, +// Start with the local 10% ramp cohort; keep the remote layer off. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 10, + [CacheLayer.REMOTE]: 0, + }, + shadow: { ramp: 0 }, + }), ); -await updateUser("123", patch); -await dialcache.invalidateRemote("user_id", "123", USER_INVALIDATION_BUFFER_MS); -``` - -Invalidation writes a Redis watermark at `{encodedNamespace:encodedKeyType:encodedId}#watermark`. Tracked Redis cache entries use the same Redis Cluster hash tag, for example `{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1`, so the value key and watermark key live in the same slot. Key components are percent-encoded before joining so delimiters inside IDs or args cannot collide with delimiters in the key format. Components may not contain `{` or `}` because those characters would corrupt the hash tag. - -The internal `:dialcache-frame-v1` suffix identifies values written with DialCache's binary protocol. Watermarks are stored as decimal timestamps. - -A cached Redis value whose Redis-created timestamp is older than or equal to the watermark is treated as stale and refreshed through fallback. `invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the greater of its existing value and Redis's current time plus the buffer. While that future window is active, an invocation that reaches the tracked Redis read treats the covered value as a miss. Native `MGET` must transfer an existing stale frame before the Node decoder can reject it, so completed reads can repeatedly pay the full stale-payload transfer during a nonzero buffer window. If a successful fallback then reaches the tracked Redis write while the watermark still fences it, the stamp script reports the write as blocked, unlinks the value key — the placeholder that write just stored, along with the logically stale frame it replaced — and DialCache suppresses the corresponding process-local population; later reads of that entry avoid retransferring its payload. The fallback value still returns to its caller. A read failure or timeout never reaches that write-side cleanup, so a large stale value can continue to consume network bandwidth and trigger `cache_read_timeout` until another completed read cleans it up or its TTL expires. Request-local memoization remains unconditional. A ramped-out invocation without shadow work does not consult the watermark; a selected shadow path for that tracked key does consult it for `C0`, `C1` when needed, and any clean-miss fill, although caller-path request-local/process-local publication remains independent. - -The bundled timestamp protocol assumes that system clocks are synchronized across every Redis node eligible for primary promotion. Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache does not detect or compensate for cross-node clock skew. If this deployment assumption is violated, failover can temporarily suppress tracked cache fills or allow a pre-invalidation value to remain readable until it expires or a later invalidation advances the watermark past its timestamp. - -Watermarks are invalidation state, not disposable cache entries. The Redis deployment must preserve them for their derived TTL: use `noeviction` or an equivalent guarantee for deployments that rely on the publication fence, and choose persistence and failover guarantees appropriate to the application's consistency requirements. A missing watermark makes tracked reads miss, but a later tracked write cannot distinguish an empty cache from lost invalidation history; it creates a new baseline watermark and can publish fallback data that the lost future watermark would have rejected. Redis replication is asynchronous by default, and DialCache does not issue `WAIT` or provide strong consistency across failover. - -Tracked writes create a baseline watermark and extend its TTL to at least the value TTL plus one minute. Neither tracked writes nor invalidation shorten a longer or persistent watermark TTL; invalidation extends it to at least the remaining future-buffer window plus one minute. There is no fixed watermark retention floor, and reads do not extend watermark lifetime. - -`futureBufferMs` must be a nonnegative safe integer no greater than 31,536,000,000 (a fixed 365-day duration). The default is zero, but zero provides no stale-publication protection once Redis time advances. Every production invalidation should pass a named, application-owned nonzero value based on that application's measured or conservatively bounded timings; there is no universally safe library value. - -Size the buffer to cover the maximum expected negative clock skew between promotion-eligible Redis nodes plus the complete interval in which stale data could still reach the Redis write: source visibility or replication lag, the full remaining tail of any fallback that may already have observed the pre-mutation value, `serializer.dump`, Redis client queue and network latency, the placeholder write and the stamp script that assigns its server timestamp, and a safety margin. Invalidate only after the source mutation commits. Underestimating this interval can allow a delayed stale fallback to repopulate Redis after the watermark window ends. Overestimating it lengthens the tracked Redis miss/write-suppression window described above, increasing fallback load and, until write-side cleanup succeeds, stale-payload transfer and read-timeout risk without publishing stale values. Each fenced write inside the window also stores its full placeholder payload before the stamp unlinks it, so a long buffer on a hot large-value key adds allocator, replication, and AOF churn the previous fence-before-store protocol never paid. A larger buffer does not delay or suppress returning fallback values to callers. - -This is a timing contract rather than a cancellation or acquisition fence: the buffer prevents stale fallback results from passing that tracked Redis write only while the configured window remains active, and it does not force a fallback to read from an authoritative source. - -Targeted invalidation is remote-only and enforced by Redis watermarks. `invalidateRemote` does not evict existing request-local or process-local entries. Strongly invalidated mutable data should disable request-local and process-local caching (or use a very short process-local TTL only when stale reads are acceptable). - -## Request coalescing - -DialCache coalesces in-flight work at the lifetime of the first active cache layer: - -- When request-local caching is enabled, same-key callers in one outermost `enable()` scope share request-scoped in-flight work before the request-local lookup. Its resolved value is then memoized for later sequential calls in that scope. -- When process-local or Redis caching is enabled, same-key callers share in-flight work within one `DialCache` instance before the first active shared layer. This is reported as `scope="process"`, still applies when request-local caching is off, and can combine leaders from separate request scopes using the same instance. - -```ts -await dialcache.enable(async () => { - // Same cold key, concurrent calls: one fallback execution, one shared result. - const [a, b] = await Promise.all([getUser("456"), getUser("456")]); -}); -``` - -With Redis configured, an instance-scoped leader that misses the process-local cache runs one bounded Redis read and, on a normal miss, the fallback/cache write; followers share its remaining read budget and await the same result. On a shadow-selected served Redis hit, only that leader can schedule detached validation, so followers do not multiply source reads. Process-local-only misses share the leader's fallback/cache write. This protects Redis and the source of truth from a thundering herd on hot keys. - -Coalescing only applies when at least one cache layer is active and the use case's resolved `coalesce` policy has not disabled it. Calls outside `enable()` are true pass-through. Calls where request-local, process-local, and Redis serving are all disabled are uncached and uncoalesced, even if shadowing independently schedules detached Redis work; same-key shadow deduplication drops duplicate jobs but does not combine caller fallbacks. Because these calls were initially enabled, the fallback deadline below still applies. - -Because coalescing is keyed by the selected or direct key, concurrent calls with the same key share the leader's execution. Any function argument or captured value omitted from the key must be safe to share this way; include inputs such as locale, auth context, or cancellation behavior when they can change the returned value or whether the underlying loader should run separately. - -The per-use-case `coalesce` boolean (default true) turns this sharing off. `coalesce: false` in a `defaultConfig` or runtime overlay disables both scopes: same-key concurrent callers each perform their own layer reads with their own full remote-read budget, their own fallback with an independent [fallback deadline](#fallback-deadlines), and their own cache writes — request-local and process-local publication is last-writer-wins, and each Redis write applies its ordinary TTL-based or watermark-fenced semantics. Request-local memoization of settled values still serves later sequential calls in the same scope. Use it when the key intentionally omits per-caller inputs that must not be shared, or when callers must not inherit a leader's failure or `FallbackTimeoutError`. Disabling coalescing reintroduces the thundering-herd exposure described above, emits `request`/`miss`/latency metrics once per caller instead of once per flight, never emits `dialcache_coalesced_counter`, and keeps `getCoalescingState()` idle for that use case. With shadow work enabled, each un-coalesced caller may attempt to schedule detached validation; same-key shadow deduplication and `shadowMaxInFlight` still bound admitted jobs and drop the excess, but source reads are no longer combined. - -### Fallback deadlines - -Once an initially enabled invocation starts its fallback, DialCache applies a 60-second monotonic deadline by default. Set `fallbackTimeoutMs` once on a cached wrapper or on each `getOrLoad()` invocation to choose a positive integer deadline in milliseconds, up to 2,147,483,647, or set it to `null` to preserve an intentionally unbounded fallback: - -```ts -import { FallbackTimeoutError } from "dialcache"; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetUserWithDeadline", - cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), - fallbackTimeoutMs: 2_000, - }, +// Later, ramp the process-local and remote layers to 100%. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 100, + [CacheLayer.REMOTE]: 100, + }, + shadow: { ramp: 0 }, + }), ); -try { - await dialcache.enable(() => getUser("123")); -} catch (error) { - if (error instanceof FallbackTimeoutError) { - logger.warn("source lookup exceeded its DialCache budget", { - useCase: error.useCase, - timeoutMs: error.timeoutMs, - }); - } -} +// Reverse the rollout without changing getUser. +runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); ``` -The timer starts only when the fallback begins, including after a remote-read deadline has elapsed. When coalescing is enabled (the default), same-key followers share the process or request-local leader's remaining budget and receive its `FallbackTimeoutError`; pass-through invocations where every layer is disabled, and callers whose use case disables `coalesce`, have independent timers. Cache hits create no fallback timer. Calls that were initially outside an enabled context remain true pass-through and are not timed out, even when the operation configures `fallbackTimeoutMs`. +In this example, the zero-ramp `defaultConfig` is the safety net: if the +provider has no matching entry, both shared serving layers and shadow work +remain off. -Deadline delivery requires the JavaScript event loop to make progress. It cannot preempt a synchronous fallback prefix or other event-loop blocking, so rejection can arrive later than the configured duration; when control returns, DialCache checks the monotonic deadline before accepting the result. The deadline timer remains referenced until the fallback settles or times out. Consequently, an abandoned enabled fallback can keep an otherwise idle short-lived process alive until that deadline; shutdown code should drain outstanding DialCache work rather than discarding its promises. +In production, the provider can read from an application-owned dynamic config +client instead of an in-memory map. DialCache resolves one policy snapshot per +enabled `cached()` or `getOrLoad()` invocation. -Timing out rejects the DialCache chain and clears its flight normally. A later fallback resolution is ignored, so that timed-out invocation cannot become the accepted `S` for a shadow fill or proceed to ordinary serializer, Redis, or local-cache publication. The underlying function is not canceled and may continue its own I/O or side effects; give the source operation its own native timeout or `AbortSignal` whenever possible. `fallbackTimeoutMs: null` disables this guard and makes finite fallback settlement entirely application-owned. Use the `null` escape hatch only after intentionally accepting that liveness risk. It does not create an unbounded detached shadow operation: [shadow validation](#shadow-validation) still uses a 60-second whole-job budget. +The process-local and remote layers each need an effective TTL. With a TTL but +no ramp, a layer defaults to `100`; a ramp of `0` disables it, `100` selects +every key, and an intermediate value selects a stable key cohort. -Timeout failures retain the bounded metrics classification `error="fallback"` with `in_fallback="true"`; the typed error provides the timeout details without adding high-cardinality labels. +Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% +of calls. Increasing or decreasing a ramp preserves membership for keys that +remain inside the threshold, and local and remote cohorts are layer-specific. +DialCache keeps the assignment stable across releases. -### Coalescing state +Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing +entries rather than deleting them; a later ramp-up can reuse entries that +remain valid. -`getCoalescingState()` returns a detached, point-in-time snapshot of process-scoped flights owned by that `DialCache` instance: +Request-local caching and in-flight sharing are controlled separately. +`requestLocal` defaults to `false`; `coalesce` defaults to `true`. +`DialCacheKeyConfig.disabled()` turns request-local caching and shadow work off +and ramps both shared layers to `0`. It leaves `coalesce` unset, so a later +runtime ramp-up returns to default-on coalescing unless policy explicitly opts +out. -```ts -const state = dialcache.getCoalescingState(); +A remote serving ramp of `0` alone does not override an inherited nonzero +`shadow.ramp`; set both ramps to `0` to stop new invocation-driven Redis reads +and fills. -state.process.activeLeaders; -state.process.activeFollowers; -state.process.oldestLeaderAgeMs; // null when idle -``` +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for sparse-overlay precedence, provider failure behavior, externally +coordinated cohorts, remote-read deadlines, and layer validation. -A leader is one exact cache key currently tracked by the instance-scoped coalescer. A follower is each later invocation that joined that pending leader; the initiating invocation is not counted as a follower. Followers remain counted until their leader settles because abandoning a JavaScript promise is not observable. Request-local flights are deliberately excluded because their lifecycle is bounded by the outer `enable()` scope. Use cases that disable `coalesce` never register process flights and never appear in the snapshot. `oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is requested. +## Validate Redis before serving it -There is no library-wide flight cap or age-based replacement. A registry cap would bound only DialCache metadata while overflow or eviction could still create unbounded source work and unsafe duplicate publication. Finite operation deadlines provide eventual cleanup; application admission control and backpressure remain responsible for bounding simultaneous distinct-key work. Monitor leader count and oldest age to verify that those budgets hold in production. +Shadow mode can exercise tracked or untracked Redis entries before Redis is +allowed to serve callers. On a selected Redis hit, DialCache returns +the cached value first, then compares a fresh decoding of the retained payload +with a detached source read. -## Metrics +When the remote serving ramp excludes a selected key, shadow work reuses the +caller's source result to inspect Redis and can fill a clean miss. -Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the constructor. `new DialCache()` does not import a metrics backend, register collectors, or emit metrics. +Shadow work never supplies, delays, or rejects the caller. It is separately +sampled by `shadow.ramp`, bounded per instance by `shadowMaxInFlight`, and +disabled unless the metrics adapter implements the shadow outcome hook. -### Prometheus +Shadow validation can add source and Redis work, remains best-effort during +shutdown, and requires a valid remote TTL. Tracked keys retain watermark +fencing; untracked shadow fills use ordinary TTL-based last-writer-wins writes. +If upgrading from before `v0.15.0`, set `shadow.ramp` to `0` first when +untracked shadow reads and fills have not yet been approved. -Install `prom-client` separately, create the registry your application owns, and pass the explicit Prometheus adapter to DialCache: +Confirmed mismatch warnings are separately opt-in through +`shadow.logMismatches`. They can include logical cache keys and JSON-serialized +values: the fields are capped, not redacted, so enable them only for trusted, +bounded values under an approved logging and data-handling policy. -```bash -pnpm add prom-client@^15.1.3 -``` +See [Shadow validation and Redis bootstrap](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) +for eligibility, rollout design, comparison semantics, command amplification, +deadlines, metrics, invalidation, and lifecycle requirements. -```ts -import { Registry } from "prom-client"; -import { DialCache } from "dialcache"; -import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; +## How the read path works -const registry = new Registry(); -const dialcache = new DialCache({ - namespace: "users-api", - metrics: createPrometheusDialCacheMetrics({ - registry, - prefix: "myapp_", // myapp_dialcache_request_counter, etc. - }), -}); +Inside an enabled scope, active layers are checked in order: -app.get("/metrics", async (_req, res) => { - res.type(registry.contentType).send(await registry.metrics()); -}); +```text +request-local -> process-local LRU -> Redis or Valkey -> source loader ``` -The adapter requires a caller-owned `Registry`; it never uses the global default registry and does not clear or otherwise own the registry lifecycle. Multiple adapters with the same registry and prefix reuse existing collectors when their type, help, labels, histogram buckets, and exemplar mode match. Adapter construction fails before registering anything if a same-name collector has an incompatible schema; use a unique prefix or a separate registry to resolve the collision. - -The Prometheus adapter emits: - -| Metric | Type | Labels | Description | -| --- | --- | --- | --- | -| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips (`context`, `policy_disabled`, `invalid_ttl`, `invalid_ramp`, `ramped_down`, `config_error`) | -| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors classified by a bounded failure site | -| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by `request_local` or `process` scope | -| `dialcache_shadow_validation_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `outcome` | Sampled Redis shadow-job outcomes | -| `dialcache_compression_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `outcome` | Payload compression outcomes: writes record `compressed`, `below_threshold`, `not_smaller`, or `write_over_limit`; reads record `decompressed`, `fallback_raw`, or `read_over_limit` | -| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency | -| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes, before compression | -| `dialcache_stored_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Stored Redis payload size in bytes, after compression and escaping | -| `dialcache_compression_ratio_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Compressed-to-original payload size ratio for compressed writes | -| `dialcache_compression_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Payload compression and decompression latency in seconds | - -`policy_disabled` means that a process-local or Redis layer has no effective TTL after runtime overlays are applied. It is an intentional policy outcome, including the default when `defaultConfig` is omitted, rather than a configuration-loading failure. - -Every metric carries `cache_namespace`, including disabled-context, key-construction, coalescing, shadow-validation, and invalidation paths that do not have a constructed key. Its value is `DialCacheConfig.namespace`, defaulting to `urn`. The `layer` label is `request_local`, `local` (process-local), `remote` (caller-serving Redis), or `remote_shadow` (detached, non-serving Redis work); `noop` means no cache layer was reached. Detached reads, serializer work, payload sizes, and Redis read/write errors use `remote_shadow`, while the dedicated bounded shadow `outcome` records the terminal job result. The bounded `scope` label on `dialcache_coalesced_counter` distinguishes request-local from instance-scoped single-flight work. `scope="process"` coordinates calls only within one `DialCache` instance; separate instances in the same process do not share in-flight state. - -### Datadog - -Install `hot-shots` separately, create the DogStatsD client your application owns, and pass it to the Datadog adapter: - -```bash -pnpm add hot-shots@^17.0.0 -``` +The wrapped function or inline loader is the fallback and remains the source of +the returned value when every active layer misses or a cache operation fails +open. + +- A request-local hit returns the value memoized in the current outermost + `enable()` scope. +- A process-local hit returns from the `DialCache` instance's bounded LRU. +- A process-local miss can read Redis and populate the process-local cache. +- A remote miss runs the fallback and attempts to populate the active cache + layers. +- Selected tracked or untracked keys can schedule detached shadow work after a + Redis serving hit or when the Redis serving ramp excludes the key. Shadow + work never serves the caller; it can validate a hit or fill a clean miss. +- A caller-path remote read failure or timeout runs the fallback without a + second caller-path Redis operation. Tracked invalidation adds a stricter + [publication rule](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md#read-and-write-behavior). +- Same-key concurrent work is coalesced within the scope of the first active + layer unless the resolved policy sets `coalesce: false`. + +When all serving layers are disabled by policy, an initially enabled call +remains uncached and uncoalesced even if selected shadow work runs +independently; its fallback deadline still applies. A call that started outside +an enabled scope remains a true pass-through and does not get a DialCache +deadline or shadow work. + +## Core concepts + +### Cache operations and keys + +`cached(fn, options)` defines both a callable and the value-identity contract: + +| Option | Required | Purpose | +| --- | --- | --- | +| `keyType` | yes | Names the kind of id and, with `id`, the invalidation unit for tracked Redis entries. | +| `useCase` | yes | Identifies this individual cache in stored keys and metrics. | +| `cacheKey` | yes | Selects the bare id or `{ id, args }` from the function parameters. | +| `defaultConfig` | no | Supplies the baseline policy overlaid by runtime config. | +| `serializer` | for statically non-JSON return types | Defines the Redis representation for this operation's value. | +| `shadowComparator` | no | Defines synchronous application-level equality for shadow validation; strict deep equality is the default. | +| `trackForInvalidation` | no | Opts the remote entries into watermark-based targeted invalidation. | +| `fallbackTimeoutMs` | no | Sets the fallback deadline; defaults to `60_000`, and `null` disables it. | + +Use `getOrLoad(load, options)` when a one-shot calculation should remain inline. +It follows the same cache, policy, coalescing, invalidation, serialization, and +deadline contracts, but takes a direct `key` instead of a `cacheKey` selector. +For repeated inline calls that represent the same operation, reuse one stable, +deployment-defined `useCase`. + +The selected or direct key must include every input dimension that can affect +the returned value. Same-key concurrent calls may share the leader's execution, +so ignored function arguments or captured values such as auth context, locale, +or cancellation behavior must truly be safe to share. + +Set a stable, application-specific `namespace` when applications or +environments share Redis: ```ts -import StatsD from "hot-shots"; -import { DialCache } from "dialcache"; -import { createDatadogDialCacheMetrics } from "dialcache/datadog"; - -const dogStatsD = new StatsD({ - host: process.env.DD_AGENT_HOST, - globalTags: { service: "users-api", env: process.env.DD_ENV ?? "development" }, - errorHandler: (error) => logger.warn("DogStatsD error", { error }), -}); - const dialcache = new DialCache({ - namespace: "users-api", // cache identity and cache_namespace tag - metrics: createDatadogDialCacheMetrics({ - client: dogStatsD, - observationMetricType: "distribution", - namespace: "dialcache", // metric-name prefix: dialcache.request.count, etc. - }), + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, }); - -// After outstanding cache operations finish during application shutdown: -dogStatsD.close(); ``` -`hot-shots` is the supported and tested client, but the adapter depends only on the exported `DatadogDogStatsDClient` structural interface. DialCache does not import or install `hot-shots`, create a client, flush buffers, close sockets, or otherwise own the client lifecycle. - -`observationMetricType` is required. `"distribution"` is recommended when latency and size percentiles must aggregate across hosts; enable the desired distribution percentiles and aggregations in Datadog. Choose `"histogram"` when host-level histogram aggregation matches your existing Datadog setup. The choice applies uniformly to every duration, size, and ratio metric. Both modes produce Datadog custom metrics. Distribution volume scales with unique tag-value combinations: Datadog counts five baseline aggregations per combination, and enabling percentile aggregations adds five more. Review [Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) before rollout. Do not send both types under the same namespace: when changing types, use a new namespace during migration so one metric identity never mixes histogram and distribution points. - -`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to `dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache namespace emitted as the `cache_namespace` tag. The Datadog metric namespace must start with a letter and contain only letters, numbers, underscores, and dot-separated non-empty segments. The adapter rejects invalid metric namespaces and final metric names longer than 200 characters rather than relying on client-side normalization. A `hot-shots` `prefix` is applied after the adapter constructs the name, so include that prefix when checking the final length and avoid combining it with the metric namespace accidentally. Client-level `globalTags` are appended by `hot-shots`; the table below lists the tags added by the adapter. +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for key encoding, secondary arguments, namespace changes, serializers, and +value ownership. -The Datadog adapter emits exact increments of `1` for counters and preserves seconds and bytes without unit conversion: +### Cache layers -| Metric | Type | Tags | Description | -| --- | --- | --- | --- | -| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | -| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors by bounded failure site | -| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | -| `dialcache.shadow.count` | Count | `cache_namespace`, `use_case`, `key_type`, `outcome` | Sampled Redis shadow-job outcomes | -| `dialcache.compression.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `outcome` | Payload compression outcomes: writes record `compressed`, `below_threshold`, `not_smaller`, or `write_over_limit`; reads record `decompressed`, `fallback_raw`, or `read_over_limit` | -| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | -| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes, before compression | -| `dialcache.stored.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Stored Redis payload size in bytes, after compression and escaping | -| `dialcache.compression.ratio` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Compressed-to-original payload size ratio for compressed writes | -| `dialcache.compression.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Payload compression and decompression latency in seconds | - -Observer throws and rejections from returned promises or thenables are isolated by DialCache's fail-open metrics boundary. Buffered transport failures that are not represented by a returned thenable happen outside that boundary, so configure the DogStatsD client's error handling and shutdown behavior as part of application ownership. - -### Error categories - -The `error` label reports where an operation failed rather than copying the thrown value's class or `Error.name`: - -| `error` | Meaning | -| --- | --- | -| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | -| `config_resolution` | Runtime or layer configuration, or ramp resolution, failed | -| `cache_read` | A local-cache or Redis read failed | -| `cache_read_timeout` | A Redis read exceeded its effective remote-read deadline | -| `cache_write` | A local-cache or Redis write failed; tracked Redis writes add a benign self-healing floor under same-key contention (see [Redis-backed TTL cache](#redis-backed-ttl-cache)) | -| `serialization_load` | Deserializing a Redis payload failed | -| `serialization_dump` | Serializing a value for Redis failed | -| `compression` | zstd compression failed while preparing a Redis write | -| `invalidation` | Writing an invalidation watermark failed | -| `fallback` | The wrapped application function failed or exceeded its DialCache deadline | -| `unknown` | Reserved for an otherwise unclassified future failure site | - -These values are defined by the backend-neutral core and are identical for every metrics adapter. Raw thrown values, error names, messages, timeout values, cache IDs, arguments, and Redis keys are never included in metric labels. Operational errors are still passed to the configured logger where the existing failure path logs them. `in_fallback` remains the explicit cache-plumbing-versus-application distinction. - -### Custom adapters - -For other telemetry backends, implement `DialCacheMetricsAdapter` and pass the adapter through `new DialCache({ metrics })`. Every backend-neutral label object exposes the logical namespace as camel-case `cacheNamespace`; adapters should map it to their backend's `cache_namespace` label/tag. This field is present even when no key or cache layer was reached. Implement the optional `shadowValidation` method to enable shadow work as well as record its outcomes; omitting it leaves all shadow work disabled even when `shadow.ramp` is nonzero or mismatch logging is enabled. Every metrics callback is fire-and-forget: DialCache isolates synchronous throws and consumes rejections from returned promises or thenables, but never awaits or drains observer work. Omit `metrics` to disable metrics. +| Layer | Scope | Primary use | +| --- | --- | --- | +| Request-local | Outermost `enable()` scope | Memoize repeated reads during one bounded request or job. | +| Process-local | One `DialCache` instance | Serve hot values from a bounded in-process LRU. | +| Redis or Valkey | Shared remote store | Reuse TTL-cached values across processes and hosts. | -## Maintainers +Each invocation uses one resolved policy snapshot for all three layers. +Request-local storage has no capacity limit, so use it only for short-lived +scopes with bounded key cardinality. Process-local values count toward one +instance-wide entry cap. Remote values use a serializer selected by the cache +operation or the Redis configuration. -### Cache-path benchmark +Redis payloads at least 4 KiB are compressed with zstd by default, but only +when the encoded value becomes smaller. Compression runs synchronously on the +Node.js event loop, so tune the threshold for your payload and latency profile +or set `redis.compression` to `false`. -From a repository checkout, run the semantic microbenchmark after installing dependencies: +Reads still decode marked compressed entries after writes are disabled, which +means `compression: false` does not strand entries for readers that understand +the envelope. Older package versions are a separate mixed-deployment and +rollback concern. See +[Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md#compression) +for configuration, metrics, size limits, and custom-serializer compatibility. -```bash -pnpm benchmark:request-local -``` +Shadow validation uses detached Redis work but is not another serving +`CacheLayer`. Its sampling, capacity, deduplication, deadline, and metrics +contracts are independent of request coalescing and the remote serving ramp. -The command builds `dist` before reporting ten scenarios: sequential request-local hits, sequential process-local hits, enabled bounded fallbacks, request-local coalescing fan-out, process coalescing fan-out, remote-read-deadline coalescing fan-out, tracked Redis hits with shadow omitted, tracked Redis hits deterministically outside a partial shadow ramp, a ramped-down warm-hit confirmation, and a ramped-down clean-miss fill. Both shadow scenarios prove that the caller completes before detached Redis work. The benchmark is a maintainer tool and is not included in the published package. It asserts fallback counts, Redis behavior, coalescing state, timer cleanup, returned values, exactly-once SoT reuse, and conditional confirmation/fill without applying a timing threshold. Override its work sizes with `DIALCACHE_BENCH_ITERATIONS` and `DIALCACHE_BENCH_FANOUT`. +Cached in-memory values are shared by reference. Treat every returned value as +immutable, or copy it explicitly before mutation. -### Redis write benchmark +### Targeted invalidation -With a Redis reachable at `REDIS_URL` (default `redis://127.0.0.1:6379`, e.g. `docker run --rm -p 6379:6379 redis:6.2`), measure the local build's write path: +Mutable Redis-backed use cases can opt into watermark-based invalidation with +`trackForInvalidation: true`, then call: -```bash -pnpm benchmark:redis-write +```ts +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); ``` -The command builds `dist`, then runs sequential tracked and untracked writes at 100 B, 10 KiB, 100 KiB, and 1 MiB payloads, reporting server-side command cost per write from `INFO commandstats` (the `EVALSHA` entry envelopes the stamp script's internal calls) and client-side p50/p95 latency. Like the cache-path benchmark it is a maintainer tool, is not part of the published package, and asserts no timing thresholds — absolute numbers depend on the machine, engine, and load, so compare runs only within one environment. Scale iteration counts with `DIALCACHE_BENCH_WRITE_SCALE`. - -### Releasing - -Publishing starts by manually running the `Release` workflow from current `main`. After the package checks pass, Semantic Release selects the next version from Conventional Commits since the highest stable `vX.Y.Z` tag. While the package is pre-1.0, breaking changes bump minor — their `BREAKING CHANGE:` footers still drive full release notes without forcing 1.0.0 — `feat` bumps minor, and every other normal PR-title type (`fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, `chore`, `ci`, and `revert`) bumps patch. The highest required bump wins. Major bumps return when 1.0.0 is cut; `release.config.mjs` implements this table and must change together with this section. - -The workflow opens a `release: ` PR whose only change is the matching `package.json` version. `release` is a reserved Conventional Commit type configured not to request another release, so the version-control commit does not cause an extra bump. GitHub marks workflow runs for a PR opened with `GITHUB_TOKEN` as approval-required; approve those runs, review the PR, and squash-merge it normally through the protected branch. - -The merge triggers the publish job. Before any release side effect, it verifies current `main`, the release commit subject, the one-file diff, the package version, the absent tag, and Semantic Release's independently calculated version and commit. It then reruns the package checks and asks Semantic Release to create the matching Git tag, publish the public npm package with provenance, and publish the GitHub release. - -The repository must enable **Allow GitHub Actions to create and approve pull requests** under Actions workflow permissions. This workflow uses that capability only to create the version PR; it never approves or merges one, and no ruleset bypass actor or persistent release credential is required. +`invalidateRemote()` is an explicit remote maintenance operation and requires +this `DialCache` instance to have a Redis or Valkey client. Without one, it +rejects instead of reporting a no-op as successful. + +Invalidation is deliberately remote-only. It does not evict existing +request-local or process-local values, so strongly invalidated mutable data +should disable those layers or tolerate their TTL-bounded staleness. + +The buffer must be a named, application-owned nonzero value no greater than +365 days, sized for clock skew and the full stale-work window. See +[Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) +before enabling it in production. + +### Request coalescing and fallback deadlines + +Concurrent callers with the same cache key share active work within the first +active cache scope: one outer request for request-local caching, or one +`DialCache` instance when process-local or remote caching is active. This +mitigates hot-key stampedes inside that scope; it is not cross-process +coordination. + +Same-key followers share the leader's remaining remote-read budget. The +fallback deadline starts separately only if and when the source loader begins. + +Set `coalesce: false` only when concurrent calls with the same cache key must +not share caller-specific execution. Each caller then performs independent +layer reads, fallback work, deadlines, and writes; settled request-local values +can still serve later sequential calls. This also gives up stampede protection +for that use case. + +Enabled fallbacks have a 60-second monotonic deadline by default. Timing out +rejects the DialCache chain and prevents the late result from being published, +but it does not cancel the underlying function. Give source operations their +own native timeout or `AbortSignal`. + +See [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) +for exact sharing, deadline, cleanup, and admission-control contracts. + +### Observability + +Metrics are disabled unless a `DialCacheMetricsAdapter` is supplied. First-party +adapters support caller-owned Prometheus registries and Datadog DogStatsD +clients. Their fixed schemas report layer requests, misses, disabled reasons, +coalescing scopes, serialization and compression work, shadow outcomes, and +cache versus fallback failures. + +Keep application-owned namespaces, use-case names, and key types stable and +low-cardinality. Optional confirmed-mismatch warnings are value-bearing logs, +not metrics, and require separate data-handling review. + +See [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) +for installation, collector schemas, metric names, and custom adapters. + +## Production checklist + +Before ramping a use case: + +- enable DialCache only around read paths, and keep mutation paths inside + `disable()` or outside the enabled boundary; +- verify that every selected or direct key includes each value and execution + dimension that is unsafe to share; +- begin at `0` or a small deterministic key cohort, monitor source load, cache + errors, hit rate, latency, remote-read and fallback timeouts, and coalescing + state, then increase in controlled steps; +- keep a runtime path to `DialCacheKeyConfig.disabled()`; +- choose an effective DialCache remote-read deadline, and configure + resource-native budgets for the underlying Redis work, config providers, + serializers, and source operation; +- use a conservative `localMaxSize` and bounded request-local scopes; +- treat cached values as immutable; +- verify serializer compatibility across mixed application versions; +- benchmark synchronous compression for representative payloads and monitor + compression outcomes, prepared payload size, and ratio before changing its + threshold; +- set `coalesce: false` only when independent same-key execution is required + and the resulting source and Redis fan-out is acceptable; +- before enabling shadow mode, confirm the loader is safe for an extra + observational read, preserve immutable inputs and results, bound concurrency, + and monitor added load and outcomes; +- before enabling shadow mismatch logging, approve how logical keys and + serialized values are redacted, transported, accessed, and retained; +- plan shutdown around detached shadow work and application-owned dependencies; and +- for tracked invalidation, use authoritative primary reads, synchronized Redis + clocks, durable non-evictable watermarks, and an application-sized nonzero + buffer. + +## Reference guides + +- [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) — definitions, keys, + runtime overlays, request-local and process-local behavior, and value + ownership. +- [Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) — node-redis and GLIDE setup, lifecycle, + liveness, native command protocol, serialization, and compression. +- [Shadow validation and Redis bootstrap](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) — + non-serving rollout, eligibility, comparison, clean-miss filling, capacity, + deadlines, metrics, mismatch diagnostics, and lifecycle. +- [Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) — watermarks, Redis Cluster + placement, clock assumptions, and buffer sizing. +- [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) — sharing scopes, + deadlines, state inspection, cleanup, and backpressure. +- [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) — Prometheus, Datadog, metric schemas, + error categories, and custom adapters. +- [Maintainer guide](https://github.com/lan17/DialCache/blob/main/docs/maintainers.md) — benchmarks and the protected release + workflow. + +DialCache is licensed under the +[MIT License](https://github.com/lan17/DialCache/blob/main/LICENSE). diff --git a/docs/coalescing.md b/docs/coalescing.md new file mode 100644 index 0000000..882d81a --- /dev/null +++ b/docs/coalescing.md @@ -0,0 +1,336 @@ +# Coalescing and fallback liveness + +[Back to the README](../README.md) + +By default, DialCache shares same-key in-flight work within the lifetime of the +first active cache layer. A per-use-case policy can disable that sharing. Each +active remote read has a finite deadline, and a separate default deadline begins +when an initially enabled invocation starts its fallback loader. + +These mechanisms reduce duplicate source work. Their deadlines help flights +settle, but eventual cleanup still requires finite application-owned budgets +for every injected operation. They do not replace cross-process coordination, +source-native cancellation, admission control, or backpressure. + +Detached [shadow work](shadow-validation.md) has a separate instance-level +registry and capacity limit. It is not another coalescing scope. + +## Request coalescing + +DialCache has two sharing scopes. + +### Request-local scope + +When request-local caching is active and coalescing is enabled, callers with the +same key in one outermost `enable()` scope share in-flight work before the +request-local lookup. + +The resolved value is memoized for later sequential calls in that scope. A +different outer request has a different request-local flight registry. + +### Process scope + +When process-local or remote caching is active and coalescing is enabled, +same-key callers share work within one `DialCache` instance before the first +active process-local or remote layer. + +This is reported as `scope="process"`, but it is instance-scoped: + +- separate requests using the same `DialCache` instance can share; +- separate `DialCache` instances in one process do not share; and +- separate processes or hosts do not share. + +```ts +await dialcache.enable(async () => { + // Same cold key and active process-local or remote layer: + // one fallback execution, one shared result. + const [first, second] = await Promise.all([ + getUser("456"), + getUser("456"), + ]); +}); +``` + +With a remote layer configured, an instance-scoped leader that misses +process-local cache performs one bounded Redis read. Followers share that read +and its remaining deadline. On a remote miss, the leader runs the fallback and +cache write; followers await that result. + +For a process-local-only miss, followers share the leader's fallback and local +write. This mitigates a thundering herd on one hot key within the instance. + +### Per-use-case opt-out + +`DialCacheKeyConfig.coalesce` is a sparse runtime boolean whose effective +default is `true`. Set it to `false` in a use case's `defaultConfig` or runtime +overlay to disable both request-local and process-scoped single-flight: + +```ts +import { CacheLayer, DialCacheKeyConfig } from "dialcache"; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUserWithoutSingleFlight", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + coalesce: false, + }), + }, +); +``` + +Concurrent same-key callers then each perform: + +- their own active-layer reads with a full independent remote-read budget; +- their own fallback, error, and fallback deadline when a fallback is needed; +- their own cache writes after a miss. + +Request-local and process-local publication is last-writer-wins. Each Redis +write keeps its ordinary TTL-based or watermark-fenced semantics. A settled +request-local value can still serve a later sequential call in the same outer +scope; the policy disables in-flight sharing, not memoization or cache hits. + +Runtime overlays can explicitly change the field in either direction. Omission +inherits the baseline and ultimately defaults to `true`. +`DialCacheKeyConfig.disabled()` deliberately leaves `coalesce` unset: with every +serving layer off there is no flight to share, and a later runtime ramp-up +coalesces again unless it explicitly opts out. + +The public constructor and static `defaultConfig` validation require a boolean +when the field is present. A malformed runtime value fails config resolution for +the whole invocation: DialCache warns, records `config_resolution` and +`config_error`, and executes the fallback uncached without touching Redis. + +Use the opt-out when executions with the same value identity must not inherit a +leader's failure, cancellation behavior, or `FallbackTimeoutError`. It does not +make an incomplete cache key safe: if an input changes the returned value, put +it in the key or disable the affected cache layers. Disabling coalescing +reintroduces thundering-herd exposure, independent Redis load, and write races. + +No metric or state surface is added. An opted-out use case emits no +`coalesced` event, records request, miss, and latency observations once per +caller rather than once per flight, and does not register process state in +`getCoalescingState()`. + +## When calls do not coalesce + +Coalescing applies only when at least one cache layer is active and the resolved +`coalesce` policy is not `false`: + +- calls that start outside `enable()` are true pass-through; +- initially enabled calls with every layer disabled are uncached and + uncoalesced; +- a use case with `coalesce: false` keeps each caller's cache path independent; +- process-scoped work is never shared across `DialCache` instances. + +An initially enabled all-disabled call still receives the fallback deadline +described below. + +The full constructed cache key always defines cached-value identity. Include +locale, auth context, or any other input that can change the returned value, +regardless of the coalescing policy. + +When coalescing is enabled, that same key also defines execution identity: +concurrent calls with the same key share the leader's execution. Include +cancellation behavior and other execution-only inputs when they must differ by +key, or use `coalesce: false` when their results remain safe to cache under the +same value identity but their in-flight work must stay independent. + +### Shadow work does not enable caller coalescing + +Shadow admission does not make an otherwise all-disabled caller path +coalesced. + +When the remote layer is the only configured serving layer and its ramp +excludes a key, concurrent calls each run their own source fallback. Same-key +shadow jobs are deduplicated by admitting one and reporting the others as +`dropped`; callers do not join or await that job. + +A serving Redis hit reached through a process-scoped leader schedules at most +one shadow job for its coalesced followers. With `coalesce: false`, each caller +can attempt to schedule validation, but exact-key shadow deduplication admits at +most one concurrent job and reports the other attempts as `dropped`. + +`shadowMaxInFlight` limits scheduled or running shadow jobs across the +instance, independently of request-local and process-scoped flights. See +[Shadow validation and Redis bootstrap](shadow-validation.md) for the full +admission and lifecycle contract. + +## Fallback deadlines + +Once an initially enabled invocation begins its wrapped fallback, DialCache +applies a 60-second monotonic deadline by default. + +Set `fallbackTimeoutMs` on a cached wrapper or `getOrLoad()` invocation to +choose a positive integer deadline in milliseconds, up to 2,147,483,647. Set +it to `null` only when the application intentionally accepts an unbounded +fallback: + +```ts +import { FallbackTimeoutError } from "dialcache"; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUserWithDeadline", + cacheKey: (userId) => userId, + defaultConfig: DialCacheKeyConfig.enabled(60), + fallbackTimeoutMs: 2_000, + }, +); + +try { + await dialcache.enable(() => getUser("123")); +} catch (error) { + if (error instanceof FallbackTimeoutError) { + logger.warn("source lookup exceeded its DialCache budget", { + useCase: error.useCase, + timeoutMs: error.timeoutMs, + }); + } + throw error; +} +``` + +### When the timer runs + +The timer starts only when the fallback begins: + +- same-key followers share the request-local or process leader's remaining + budget and receive its `FallbackTimeoutError`; +- callers with `coalesce: false` start independent fallback timers and receive + independent errors; +- a remote read failure or timeout starts the fallback timer only when the + source loader begins; +- enabled pass-through invocations where every layer is disabled have + independent timers; +- cache hits create no fallback timer; and +- calls that began outside an enabled context remain true pass-through and are + not timed out, even when the operation has `fallbackTimeoutMs`. + +The fallback deadline does not cover work that happens before fallback. An +active remote read has its own resolved +[remote-read deadline](redis.md#remote-read-deadlines-and-async-liveness), while +a pending config provider or serializer load does not. Serialization and a +Redis write after fallback also remain outside it. Give every injected +operation its own finite, resource-native budget. + +### Event-loop behavior + +Deadline delivery requires the JavaScript event loop to make progress. It +cannot preempt a synchronous fallback prefix or other event-loop blocking. +Rejection can therefore arrive later than the configured duration. + +When control returns, DialCache checks the monotonic deadline before accepting +the result. The timer remains referenced until the fallback settles or times +out. An abandoned enabled fallback can keep an otherwise idle short-lived +process alive until the deadline. + +Shutdown code should await outstanding caller-path DialCache promises rather +than discard them. This does not drain detached shadow jobs; see +[Redis lifecycle ownership](redis.md#lifecycle-ownership) for dependency +shutdown requirements. + +### Timeout does not cancel the source + +Timing out: + +1. rejects the DialCache chain; +2. clears its coalescing flight normally, when one exists; +3. ignores a later fallback resolution; and +4. prevents that invocation from proceeding to serializer, Redis, or local + publication. + +The underlying loader is not canceled and may continue its own I/O or side +effects. Give the source operation a native timeout or `AbortSignal` whenever +possible. + +`fallbackTimeoutMs: null` disables the guard and makes finite fallback +settlement entirely application-owned. Use that escape hatch only after +intentionally accepting the liveness risk. + +Timeout failures retain the bounded metrics classification +`error="fallback"` with `in_fallback="true"`. The typed error carries timeout +details without adding high-cardinality labels. + +A shared remote-read timeout emits one `cache_read_timeout` error for the +leader, not one per follower. With coalescing disabled, each caller owns its +read and can emit its own timeout error. + +### Shadow deadlines are separate + +A finite `fallbackTimeoutMs` also supplies the whole-job deadline for detached +shadow work. Setting it to `null` removes the caller fallback deadline, but +shadow work still uses the 60-second default. + +For a served Redis hit, the shadow clock starts when detached validation +begins. For a remote-ramped-down call, it starts before the caller's source +operation, so synchronous source work consumes the same budget. Shadow work +never delays or rejects the caller. + +The shadow scheduler and deadline timer are unreferenced. A deadline prevents +later serialization or write dispatch, but cannot cancel an already-started +source call, serializer, raw Redis read, or dispatched Redis write. + +Underlying shadow-owned work that has already started can retain a capacity +slot until it settles, even after the bounded outcome is reported. A +caller-owned source promise reused by a ramped-down shadow path is the +exception: by itself, it stops retaining that slot at the shadow deadline. + +## Inspecting process-scoped flights + +`getCoalescingState()` returns a point-in-time copy of caller-path +process-scoped flights owned by one `DialCache` instance: + +```ts +const state = dialcache.getCoalescingState(); + +state.process.activeLeaders; +state.process.activeFollowers; +state.process.oldestLeaderAgeMs; // null when idle +``` + +A leader is one exact cache key currently tracked by the instance-scoped +coalescer. A follower is each later invocation that joined that pending leader; +the initiating invocation is not counted as a follower. + +Followers remain counted until their leader's DialCache promise settles, +including by deadline rejection. The underlying source operation may continue +after that point. + +Request-local flights are deliberately excluded because their lifecycle is +bounded by the outer `enable()` scope. Shadow jobs are also excluded; they use +their own capacity registry and outcome metrics. +Use cases with `coalesce: false` never register process flights and therefore do +not appear in this state. +`oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is +requested. + +## Admission control remains application-owned + +There is no library-wide cap or age-based replacement for caller-path +request-local or process-scoped flights. `shadowMaxInFlight` bounds only +detached shadow jobs. + +A registry cap would bound only DialCache metadata. Overflow or eviction could +still create unbounded source work and unsafe duplicate publication. +DialCache's remote-read and fallback deadlines cover only those phases; +provider, serializer, and Redis-write settlement remains application-owned. +Admission control and backpressure remain responsible for bounding +simultaneous distinct-key work. + +Monitor: + +- active leader count; +- active follower count; +- oldest leader age; +- remote-read timeout errors; +- fallback deadline errors; and +- source concurrency and saturation. + +Use those signals to verify that application budgets and admission control hold +under production load. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..d6a5539 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,675 @@ +# Configuration and cache layers + +[Back to the README](../README.md) + +This guide covers reusable cached functions, one-shot inline loaders, cache +identity, runtime policy, coalescing policy, request-local and process-local +behavior, Redis payload-compression configuration, and cached-value ownership. +For the complete shared remote-layer contract, see +[Redis and Valkey](redis.md). + +## Defining cache operations + +### Reusable cached functions + +`cached(fn, options)` wraps a function; the wrapped callable has the same +parameters and always returns a `Promise`. + +| Option | Required | Description | +| --- | --- | --- | +| `keyType` | yes | The kind of id the key addresses, such as `"user_id"`. Together with the id, this is the invalidation unit for tracked entries. | +| `useCase` | yes | Identifies the individual cache. It is part of the stored key and a metrics label. | +| `cacheKey` | yes | Selects a bare id or `{ id, args }` from `fn`'s parameters. | +| `defaultConfig` | no | Provides the `DialCacheKeyConfig` baseline that runtime config overlays field by field. | +| `serializer` | when the return type is not statically JSON-compatible | Selects a per-function `Serializer` for Redis values; see [Serialization](redis.md#serialization). | +| `shadowComparator` | no | Defines synchronous application-level equality for [shadow validation](shadow-validation.md); Node strict deep equality is the default. | +| `trackForInvalidation` | no; default `false` | Opts this use case's Redis entries into watermark-based [targeted invalidation](invalidation.md). | +| `fallbackTimeoutMs` | no; default `60_000` | Sets the fallback deadline in milliseconds, up to 2,147,483,647. `null` disables it; see [Fallback deadlines](coalescing.md#fallback-deadlines). | + +`useCase` is validated when the function is registered. A duplicate within one +`DialCache` instance throws `UseCaseIsAlreadyRegisteredError`, and the internal +name `watermark` throws `UseCaseNameIsReservedError`. + +### One-shot inline loaders + +`getOrLoad(load, options)` runs one zero-argument synchronous or asynchronous +loader through the same cache layers, runtime policy, coalescing, invalidation, +metrics, serialization, and deadline behavior as `cached()`. Cache-plumbing +failures fall through to the loader; loader failures still reject and clear +their tracked flight: + +```ts +const profile = await dialcache.enable(() => + dialcache.getOrLoad( + async () => { + const user = await db.getUser(userId); + return renderProfile(user, locale); + }, + { + keyType: "user_id", + useCase: "BuildProfile", + key: { id: userId, args: { locale } }, + defaultConfig: DialCacheKeyConfig.enabled(60), + }, + ), +); +``` + +The options match `cached()` except that the direct `key` replaces the +`cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and +snapshotted for each invocation. Outside an enabled scope, DialCache calls +`load` directly without constructing a key or resolving runtime policy. + +`getOrLoad()` does not register its `useCase` or detect duplicates, but it still +rejects the reserved internal name `"watermark"`. + +Repeated calls should reuse one stable, deployment-defined name such as +`"BuildProfile"`. Never derive it from a user, request, id, or other +high-cardinality input because it is part of both cache identity and metrics +labels. Put those values in `key` instead. + +Every captured value that can change the result belongs in the bare id or +`{ id, args }` key. By default, concurrent same-key calls may share one +caller's in-flight loader and cached value, so all call sites for that identity +must also agree on value meaning and serialization. + +A use case can explicitly set `coalesce: false` when its callers must execute +independently, but that does not make an incomplete cache key safe for settled +cache hits. + +Shadow work can run the loader later, after the caller has continued. Snapshot +mutable arguments or captured state before invoking the operation so that the +detached source read still represents the selected key. See +[Shadow validation and Redis bootstrap](shadow-validation.md). + +Prefer `cached()` for reusable loaders and `getOrLoad()` for calculations +intentionally local to one call site. + +## Enable and disable scopes + +DialCache performs cache work only inside an enabled asynchronous scope. Most +services create one instance and reuse it for the service process. Each +instance owns one process-local LRU, one process-coalescing registry, and one +shadow deduplication and capacity registry. Create separate instances only +when those resources should be isolated: + +| API | Behavior | +| --- | --- | +| `enable(fn)` | Enables caching for `fn` and the asynchronous work it awaits. The outermost call owns any request-local state. | +| `disable(fn)` | Temporarily restores pass-through behavior, commonly around nested mutation work. It does not evict existing values. | +| `isEnabled()` | Reports whether the current asynchronous call chain is inside a live enabled scope. | +| `withEnabled(fn)` | Exact alias for `enable(fn)`. | +| `withDisabled(fn)` | Exact alias for `disable(fn)`. | + +All five methods are instance-scoped. `enable()` and `disable()` always return a +`Promise`, including when their callback returns synchronously. Nested scopes +restore the previous state when their callbacks settle, and a nested +`enable()` inside `disable()` can opt a smaller read region back in. + +Enabled state follows Node's `AsyncLocalStorage`; it is not a process-global +flag. Once the outermost `enable()` callback settles, detached asynchronous work +that inherited the old context becomes pass-through and cannot repopulate its +closed request-local state. + +The root-exported `DialCacheContext` exposes the lower-level +`enable()`, `disable()`, and `isEnabled()` context primitive. It does not attach +itself to a `DialCache` instance or perform cache work. Most applications should +use the methods on `DialCache`. + +Keep mutation work outside the enabled boundary or inside `disable()`. Because +disabling does not evict existing values, mutable data still needs an +appropriate TTL or [targeted invalidation](invalidation.md) policy. + +## Keys, ids, and extra dimensions + +For `cached()`, the required `cacheKey` selector receives the wrapped +function's inferred parameters. `getOrLoad()` accepts the same bare id or +`{ id, args }` shape directly through `key`: + +```ts +const searchPosts = dialcache.cached( + (userId: string, page: number, filter: string) => + db.searchPosts(userId, page, filter), + { + keyType: "user_id", + useCase: "SearchPosts", + cacheKey: (userId, page, filter) => ({ + id: userId, + args: { page, filter }, + }), + defaultConfig: DialCacheKeyConfig.enabled(60), + }, +); + +await dialcache.enable(() => searchPosts("u1", 2, "active")); +``` + +The selected or direct key is the value-identity contract. It must include +every input dimension that can affect the returned value. Otherwise, distinct +calls can reuse the same cached value or share the same in-flight fallback +through request coalescing. + +### Namespace + +`DialCacheConfig.namespace` is the logical cache namespace and the first +component of every key. It defaults to `"urn"`, producing keys such as +`urn:user_id:123#GetUser`. + +Set a stable application-specific value when applications or environments may +share one Redis deployment: + +```ts +const dialcache = new DialCache({ + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, +}); +``` + +That produces Redis keys beginning with `production-users-api:...`, or +`{production-users-api:...}` for invalidation-tracked values. `namespace` is +DialCache's single cache-identity and key-partitioning setting. It participates +in request-local, process-local, Redis, coalescing, deterministic ramp, +invalidation, and metrics. + +A namespace may not contain `{` or `}` because DialCache reserves those +characters for Redis Cluster hash tags. + +### Identity rules + +- **`keyType` plus `id` is the invalidation unit for tracked Redis entries.** + `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one + watermark for that user. Any tracked Redis entry with the same `keyType` and + `id` is refreshed across all `args` variants when Redis is read. Untracked + entries do not consult the watermark. Invalidation does not evict existing + request-local or process-local entries. +- **`args` are part of the cache key.** Different arguments produce different + entries, but targeted invalidation is by id rather than by argument. +- **Scalar equality is string-based.** For matching surrounding dimensions: + - numeric `1`, string `"1"`, and bigint `1n` identify the same key; and + - argument values `null` and `"null"` match, `-0` matches `0`, and an + `undefined` argument is omitted. + + If a deployment changes the logical meaning represented by a scalar, change + an explicit identity dimension such as `keyType`, `useCase`, or an argument + name or value. +- **Non-key inputs still reach the loader.** A database handle can be a normal + function parameter ignored by `cacheKey` or a value captured by a + `getOrLoad()` loader. Concurrent same-key misses share the leader's execution + unless the resolved policy explicitly sets `coalesce: false`. Do not omit + values such as `AbortSignal`, auth context, locale, or other request-scoped + inputs unless both sharing in-flight work and reusing a settled cache value + are correct. +- **Methods need a receiver.** Pass `obj.method.bind(obj)` or + `(...args) => obj.method(...args)`; a bare `obj.method` reference loses + `this`. + +### Changing a namespace + +Changing `namespace` intentionally creates a cold-cache boundary across every +layer. Old and new keyspaces do not share Redis values or invalidation +watermarks. + +During an overlapping deployment, an invalidation handled by one version is +invisible to the other. The other version can continue serving a stale tracked +value until its value TTL expires. If remote invalidation correctness matters, +a normal rolling namespace change is unsafe. + +Use a coordinated no-overlap cutover, or an operational bridge that prevents +both versions from serving remote cache across mutations. For example, +temporarily disable and clear remote caching during the transition. After the +cutover, provision for fallback and refill load, and allow old Redis keys to +expire by TTL. + +## Runtime config and ramp controls + +Instance-wide behavior is set through the `DialCache` constructor: + +| `DialCacheConfig` option | Default | Description | +| --- | --- | --- | +| `namespace` | `"urn"` | Logical cache namespace and first key component. | +| `redis` | none | `{ client, readTimeoutMs?, serializer?, compression? }`; enables the [remote layer](redis.md). Remote reads default to a 50 ms deadline, and Redis payload compression defaults to zstd level 3 at 4,096 serialized bytes. | +| `localMaxSize` | `10_000` | Global process-local entry cap. `0` disables process-local storage. Must be a nonnegative safe integer. | +| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the operation's `defaultConfig`; `null` applies no overrides. | +| `shadowMaxInFlight` | `1` | Maximum scheduled or running shadow jobs per instance. Must be a positive safe integer. There is no queue; excess jobs are dropped and measured. | +| `metrics` | disabled | A `DialCacheMetricsAdapter`; see [Observability](observability.md). | +| `logger` | `console` | Receives operational cache failures and opted-in confirmed shadow mismatch warnings through `debug`, `warn`, and `error`. | + +Per-invocation policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` +maps keyed by `CacheLayer.LOCAL` and `CacheLayer.REMOTE`, `requestLocal` and +`coalesce` booleans, an optional `remoteReadTimeoutMs`, and an optional +`shadow` group. The root-exported `ShadowConfig` type defines that group's +independent `ramp` and default-off `logMismatches` leaves. + +### Baseline and overlay precedence + +Every cached definition or `getOrLoad()` invocation can provide an optional +per-use-case `defaultConfig`. That is the baseline policy. The +`cacheConfigProvider` result is a sparse field-level overlay on it. + +Enablement fields use this precedence: + +```text +runtime field -> defaultConfig field -> DialCache disabled baseline +``` + +The disabled baseline sets `requestLocal` to `false`, leaves the process-local +and remote TTLs unset, and leaves `shadow` absent. Coalescing defaults to +`true`, but no flight exists while every cache layer is inactive. + +Either serving layer is disabled by policy when it has no effective TTL. With +an effective TTL but no effective ramp, that layer defaults to a 100% ramp. +Shadow work remains off unless `shadow.ramp` is explicitly greater than zero. + +The remote-read deadline has two additional fallbacks: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +This value bounds how long DialCache waits for an active Redis or Valkey read. +It can be tuned per use case at runtime, but it cannot be disabled. + +`DialCacheKeyConfig` preserves omitted `requestLocal` and `coalesce` leaves as +`undefined`, so the overlay can distinguish omission from an explicit +`false`. Their effective defaults are `false` for request-local memoization and +`true` for coalescing. + +A provider result of `null`, or a defensive `undefined`, applies no overrides. +An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the +baseline. + +Overlay merging is sparse at each leaf. Top-level `requestLocal`, `coalesce`, +and `remoteReadTimeoutMs` leaves merge independently. The local and remote +entries inside `ttlSec` and `ramp` also merge independently. + +The `shadow.ramp` and `shadow.logMismatches` leaves follow the same rule. For +example, `shadow: { ramp: 0 }` disables inherited shadow admission while +preserving an inherited logging preference; `shadow: { logMismatches: false }` +suppresses warnings without changing the inherited shadow cohort. + +Use explicit values to replace inherited policy: + +- `requestLocal: false` disables request-local caching; +- `coalesce: false` gives each caller its own active layer reads, fallback + deadline, fallback execution, and cache writes; +- a process-local or remote ramp of `0` disables that serving layer; +- `shadow: { ramp: 0 }` disables new shadow work; and +- `DialCacheKeyConfig.disabled()` turns request-local and shadow work off and + ramps both serving layers to `0`. + +The remote serving and shadow cohorts are independent. A remote ramp of `0` +does not override an inherited nonzero `shadow.ramp`; set both to `0` when the +runtime policy must stop new invocation-driven Redis reads and fills. + +`DialCacheKeyConfig.disabled()` returns the complete cache-path overlay +explicitly: `requestLocal: false`, both serving ramps at `0`, +`shadow.ramp: 0`, and `shadow.logMismatches: false`. It intentionally leaves +`coalesce` unset. + +Its `ttlSec` map is empty, so inherited TTLs remain available for a later +ramp-up but inactive under this overlay. If runtime policy ramps a layer back +up, coalescing is on again unless another leaf explicitly opts out. The kill +switch does not cancel already-admitted work or disable explicit maintenance +operations such as `invalidateRemote()`. + +### Validation and snapshots + +DialCache validates `defaultConfig` when `cached()` registers a definition and +whenever `getOrLoad()` is invoked: + +- TTLs must be positive safe integers no greater than `31_536_000` seconds + (365 days); +- serving ramps and `shadow.ramp` must be finite percentages in the inclusive + range `0` through `100`; +- layer maps and `shadow` must be objects; +- `requestLocal`, `coalesce`, and `shadow.logMismatches` must be booleans when + present; and +- remote-read deadlines must be positive safe integers no greater than + 2,147,483,647 milliseconds. + +Invalid instance `redis.readTimeoutMs` or `redis.compression` values throw +during `DialCache` construction, as does an invalid `shadowMaxInFlight`. +Invalid defaults are rejected when `cached()` registers a definition or +`getOrLoad()` is invoked. +`null`, zero, fractional, non-finite, string, and larger timeout values are +invalid; remote reads have no unbounded escape hatch. + +Each registration or one-shot invocation captures an immutable internal +snapshot, including the nested `shadow` object. Mutating the supplied config or +its maps later does not change that operation's baseline. Runtime policy +changes belong in the provider's returned overlay. + +Runtime TTL and ramp leaves are used as supplied rather than falling back to +valid default leaves: + +- an invalid TTL disables that layer with `invalid_ttl`; +- a serving ramp that is nonnumeric, non-finite, below `0`, or above `100` + disables that layer with `invalid_ramp`; values are never clamped; and +- other valid layers can continue to run. + +Invalid leaves also record a `config_resolution` error, distinguishing provider +garbage from an intentional ramp-down. A malformed runtime config object, +layer-map or `shadow` shape, `requestLocal`, `coalesce`, or +`remoteReadTimeoutMs` value fails config resolution for the whole invocation. +DialCache records +`config_resolution`, marks the no-layer path `config_error`, and runs the +fallback without a Redis read or write. + +Runtime shadow leaves are isolated from caller-serving policy: + +- An invalid `shadow.ramp` records remote `config_resolution` and skips shadow + work when an otherwise eligible Redis path evaluates it. DialCache does not + clamp the value or disable valid serving layers. +- An invalid `shadow.logMismatches` preserves the cache result, shadow work, + and terminal shadow metric, but suppresses the warning and records remote + `config_resolution`. This diagnostic leaf is evaluated only after the + metrics hook, cohort, and capacity gates admit the job. + +Static invalid shadow leaves remain definition-time errors for `cached()` and +invocation-time errors for `getOrLoad()`. The former flat `shadowRamp` field is +removed rather than aliased: `DialCacheKeyConfig` and static defaults reject it +with `DialCacheKeyConfig.shadowRamp was replaced by "shadow.ramp"`; a runtime +provider result containing it fails config resolution for the whole invocation +and runs the loader uncached. + +### Provider behavior + +`cacheConfigProvider` is called for every enabled cache invocation before any +cache lookup. Keep it cheap, cache remote or config-store reads inside the +provider, and give asynchronous work a finite application-owned deadline. + +DialCache fetches and resolves one config snapshot per enabled invocation. +Provider errors do not activate defaults: they fail open, record +`config_error`, and execute the fallback uncached. + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + readTimeoutMs: 75, + }, + cacheConfigProvider: async (key) => { + if (key.useCase === "GetUser") { + return new DialCacheKeyConfig({ + // Sparse override: inherit both TTLs and the local ramp. + ramp: { [CacheLayer.REMOTE]: 25 }, + // Shadow leaves merge independently with defaultConfig.shadow. + shadow: { + // Inherit the baseline logMismatches: false. + ramp: 5, + }, + // Per-use-case override of the instance's 75 ms read deadline. + remoteReadTimeoutMs: 35, + }); + } + return null; + }, +}); + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + defaultConfig: new DialCacheKeyConfig({ + // Omitted ramps default to 100% because these layers have TTLs. + ttlSec: { + [CacheLayer.LOCAL]: 30, + [CacheLayer.REMOTE]: 300, + }, + shadow: { + ramp: 0, + logMismatches: false, + }, + }), + }, +); +``` + +Ramp values are thresholds from 0 to 100. `0` disables the layer, `100` enables +it for every key, and an intermediate value selects keys whose DialCache-owned +deterministic bucket for the full cache key and layer is below that threshold. + +For a fixed cache identity and layer, increasing a ramp only adds keys and +decreasing it only removes keys; it does not reshuffle existing membership. +Local and remote cohorts are layer-specific. + +Ramps select key cohorts, not requests or load, so a ramp of `10` does not +guarantee 10% of calls, especially for a small or skewed key population. +DialCache keeps the assignment stable across releases. + +Applications that need an externally coordinated cohort can use +`cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. + +Ramping down bypasses affected entries; it does not evict them, so a later +ramp-up can reuse entries that remain valid. + +`shadow.ramp` uses its own stable exact-key cohort, independent of both serving +ramps. Omitted and `0` disable shadow work; `100` selects every otherwise +eligible key. `shadow.logMismatches` separately opts confirmed mismatches into +byte-capped JSON warning fields; it does not enable shadow work and defaults +to `false`. Review the data-handling contract before turning it on. + +Shadowing requires a valid remote TTL and a metrics adapter with the shadow +outcome hook. Tracked and untracked Redis operations are both eligible and +keep their normal read and write mode. + +Shadow work can validate a served Redis hit or exercise Redis while the remote +serving ramp excludes the key. See +[Shadow validation and Redis bootstrap](shadow-validation.md) for eligibility, +clean-miss filling, deadlines, capacity, and rollout guidance. + +`shadowComparator` is stable operation behavior rather than runtime policy. It +defaults to Node strict deep equality and receives borrowed decoded-cache and +source values. A custom comparator must be synchronous, deterministic, +side-effect-free, non-mutating, and bounded. + +### Coalescing policy + +Coalescing is on unless the resolved policy explicitly sets +`coalesce: false`. The switch covers both request-local and instance-scoped +process flights. + +With it off, concurrent same-key callers each perform their own active layer +reads, receive a full independent remote-read and fallback budget, run their +own loader after a miss, and attempt their own writes. +Settled request-local memoization still serves later sequential calls. +Process-local and untracked Redis writes remain last-writer-wins, while tracked +Redis writes retain their watermark fence. + +Opt out when callers sharing one identity must not inherit another caller's +loader failure, timeout, or cancellation behavior. Doing so reintroduces +same-key fan-out to dependencies. + +It also suppresses coalesced-follower metrics and keeps those calls out of +`getCoalescingState()`; each caller emits its own request, miss, latency, and +error observations. See +[Coalescing and async liveness](coalescing.md) for flight scope, deadlines, +shadow scheduling, and observability details. + +### Provider key input + +`cacheConfigProvider` receives the fully constructed, read-only `DialCacheKey` +for the invocation: + +| Field | Meaning | +| --- | --- | +| `namespace` | Logical application or environment namespace. | +| `keyType` and `id` | Primary identity. The selected id has already been converted to a string. | +| `args` | Secondary dimensions as normalized, name-sorted string pairs; entries whose value was `undefined` are omitted. | +| `useCase` | Stable operation name used in cache identity and metrics. | +| `prefix` | Encoded identity prefix, including a Redis Cluster hash tag when invalidation tracking is enabled. | +| `urn` | Complete encoded cache identity, including arguments and `useCase`. | +| `defaultConfig` | The operation's snapshotted baseline policy, or `null`. | +| `serializer` | The operation-specific serializer, or `null`. | +| `trackForInvalidation` | Whether the operation uses remote watermark tracking. | + +Use the identity fields to select policy; do not derive policy names or metric +dimensions from unbounded user input. The provider result remains a sparse +overlay and must not mutate the key. + +Most applications do not construct keys directly. Custom integrations can use +the root exports: + +- `new DialCacheKey(init)` to build the same public key shape; +- `normalizeArgs(record)` to omit `undefined`, stringify scalar values, and + sort argument names; +- `invalidationPrefix(namespace, keyType, id)` to build the encoded tracked + identity; and +- `redisClusterHashTag(value)` to wrap a validated value in a Redis Cluster hash + tag. + +The namespace and hash-tag components reject `{` and `}` as described under +[Identity rules](#identity-rules). + +## Redis payload compression + +`RedisConfig.compression` is instance-wide write policy for the remote layer. +It is enabled by default when Redis is configured: + +```ts +import { DialCache, type CompressionConfig } from "dialcache"; + +const compression: CompressionConfig = { + thresholdBytes: 4_096, + level: 3, +}; + +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + compression, + }, +}); +``` + +`thresholdBytes` must be a positive safe integer and defaults to `4_096`. +`level` must be an integer from `1` through `22` and defaults to `3`. +Passing `false` disables compression for new writes; `null`, other non-object +values, and invalid leaves throw during `DialCache` construction. Compression +is static instance configuration rather than per-use-case runtime policy. + +DialCache compresses a serialized payload only when it meets the threshold and +the zstd frame plus its marker is smaller than the raw stored form. Reads +always decode marked payloads, even when writes use `compression: false`, so +turning compression off does not orphan entries already written compressed. + +Raw binary serializer output beginning with an envelope byte is escaped on +every write, including when compression is disabled. + +Compression and decompression run synchronously on the Node.js event loop. +The exact package engine range is `>=22.15.0 <23.0.0 || >=23.8.0` so +`node:zlib` exposes zstd. + +Decompressed payloads are capped at 512 MiB, and the write side refuses to +compress values above the same ceiling. Start with the default level, watch +compression duration and ratio metrics, and treat higher levels as a +latency-sensitive production change. + +See [Redis payload compression](redis.md#compression) for the exact envelope, +mixed-version rollout and rollback behavior, binary-serializer migration, and +failure semantics. See [Observability](observability.md#compression-metrics) +for the bounded outcomes and pre- versus post-compression measurements. + +## Request-local cache + +Set `requestLocal: true` to memoize resolved values for the lifetime of the +outermost `enable()` scope: + +```ts +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), + }, +); +``` + +`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled +`CacheLayer`. The provider can turn it on or off for each invocation. +`DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and remote +caching, so request-local caching must be selected explicitly. + +The resolved config applies to the whole invocation. When `requestLocal` is +false, the invocation skips request-local lookup and storage without deleting a +value already memoized in the scope. A later invocation that enables it can +reuse that value. + +The outermost `enable()` call owns the request-local lifetime; nested `enable()` +calls reuse the same scope. State is allocated lazily, so scopes that use only +process-local or remote caching do not allocate it. + +Wrap the complete Node HTTP handler so the scope matches the request: + +```ts +import { createServer } from "node:http"; + +const server = createServer((req, res) => { + void dialcache + .enable(async () => { + const user = await getUser(readUserId(req)); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(user)); + }) + .catch((error: unknown) => handleRequestError(error, res)); +}); +``` + +Request-local storage has no capacity limit, eviction, or overflow mode. Values +are retained until the outermost callback settles. Use it for short-lived +scopes with bounded key cardinality. Split long-running streams or large batch +jobs into smaller scopes. + +## Process-local cache + +The process-local layer, `CacheLayer.LOCAL`, uses one LRU per `DialCache` +instance. It keeps at most 10,000 entries by default across all use cases while +retaining each entry's configured TTL. + +Set `localMaxSize` to a nonnegative safe integer to change the global entry cap. +`0` disables process-local storage: + +```ts +const dialcache = new DialCache({ localMaxSize: 25_000 }); +``` + +The limit counts entries rather than estimating JavaScript object memory. +Recently read entries stay resident ahead of less recently used entries when +the limit is reached. + +## Cached-value ownership + +Treat values returned by cached functions or `getOrLoad()` as immutable. +DialCache does not clone or freeze values stored in request-local or +process-local memory. +Mutating a cached object can be observed by: + +- later callers in the same request; +- callers in other requests that hit the process-local cache; and +- callers that coalesced onto the same in-flight result. + +This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed +arrays, and class instances. Redis deserialization can produce a different +reference from an in-memory hit, so reference identity is layer-dependent and +is not part of the API contract. + +Copy a value explicitly before changing it: + +```ts +const sharedUser = await getUser("123"); +const editableUser = structuredClone(sharedUser); +editableUser.displayName = "New name"; +``` + +Use a narrower copy when its semantics are sufficient. The ownership boundary +remains the caller's responsibility. diff --git a/docs/invalidation.md b/docs/invalidation.md new file mode 100644 index 0000000..e964d3f --- /dev/null +++ b/docs/invalidation.md @@ -0,0 +1,301 @@ +# Targeted invalidation + +[Back to the README](../README.md) + +DialCache can invalidate related Redis entries without scanning or enumerating +keys. The mechanism is opt-in, remote-only, and based on per-identity Redis +watermarks. + +Read this complete contract before using targeted invalidation for mutable +production data. Correctness depends on cache-layer policy, Redis clock +synchronization, and an application-owned timing buffer. + +## Configure a tracked use case + +Set `trackForInvalidation: true` on a Redis-backed cached function or +`getOrLoad()` operation. After the source mutation commits, call +`dialcache.invalidateRemote(keyType, id, futureBufferMs)`: + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: dialCacheRedisClient }, +}); + +// Chosen from this application's clock-skew bound and measured +// worst-case source and fallback timings. +const USER_INVALIDATION_BUFFER_MS = 5_000; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetMutableUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + // Strongly invalidated mutable data should not use in-memory layers. + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { + [CacheLayer.REMOTE]: 300, + }, + ramp: { + [CacheLayer.REMOTE]: 100, + }, + }), + }, +); + +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); +``` + +The buffer is an application-owned safety value. DialCache cannot choose a +universally safe nonzero default. It must be a nonnegative safe integer no +greater than `31_536_000_000` milliseconds (365 days). + +`invalidateRemote()` is an explicit remote maintenance operation and requires +`DialCacheConfig.redis`. A local-only `DialCache` remains valid for normal cache +operations, but invalidation does not silently become a no-op: without Redis it +rejects a `TypeError` whose message is +`DialCache invalidateRemote requires a configured Redis client`. + +## Identity and Redis Cluster placement + +Invalidation writes a watermark at: + +```text +{encodedNamespace:encodedKeyType:encodedId}#watermark +``` + +Tracked Redis values use the same Redis Cluster hash tag. For example: + +```text +{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1 +``` + +The value and watermark therefore live in the same Redis Cluster slot. Key +components are percent-encoded before joining, so delimiters inside ids or +arguments cannot collide with delimiters in the key format. + +`namespace` may never contain `{` or `}`; tracked `keyType` and `id` values may +not contain them because those three components form the hash tag. `args` and +`useCase` are encoded outside the hash tag and may contain braces. + +The internal `:dialcache-frame-v1` suffix identifies values written with +DialCache's binary protocol. Watermarks are stored as decimal timestamps. + +`keyType` plus `id` is the invalidation unit. One watermark covers every tracked +`useCase` and `args` variant with that identity. Untracked values do not consult +it. + +## Read and write behavior + +A tracked read obtains the value and watermark in one atomic `MGET`. Bundled +cluster adapters explicitly route it to the slot primary; a standalone +node-redis client must already target the authoritative endpoint. A readable +frame whose Redis-stamped creation time is older than or equal to the watermark +is treated as stale and refreshed through fallback. + +`invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the +greater of: + +- its existing value; and +- Redis's current time plus the buffer. + +While that future window is active: + +1. A tracked Redis read receives the covered value and watermark, then treats + the value as a miss. +2. The invocation runs its fallback. +3. DialCache serializes and optionally compresses the fallback value, then a + native `SET` writes the complete payload as an unreadable placeholder. +4. A small stamp script compares Redis time with the watermark. If the window + is still active, it unlinks the placeholder and refuses publication. +5. DialCache suppresses the corresponding process-local population, while the + fallback value still returns to its caller. + +Request-local memoization remains unconditional. A ramped-out invocation +without selected shadow work does not consult the watermark and is not fenced +by it. + +This is a timing contract, not a cancellation or acquisition fence. The buffer +blocks stale fallback results from passing the tracked Redis write only while +the configured window remains active. It does not cancel the fallback or force +it to read from an authoritative source. + +If a tracked remote read rejects or exceeds its deadline, DialCache cannot +establish watermark safety. It runs the fallback but skips both the Redis write +and process-local publication. This differs from a normal tracked miss, which +can attempt the fenced Redis write. Untracked fallbacks may still populate +process-local cache, and request-local memoization remains unconditional. + +### Invalidated payload transfer and cleanup + +The atomic `MGET` transfers the complete Redis frame before the adapter can +compare its timestamp with the watermark. Large invalidated values can +therefore consume network bandwidth—and can repeatedly exceed the remote-read +deadline—even though DialCache will not serve them. + +A successful fallback that reaches the tracked stamp while the fence is active +partially mitigates this: its placeholder `SET` replaces the stale frame and +the stamp script unlinks the placeholder. Later reads then avoid transferring +the old payload. + +A read error or timeout skips the write entirely, so it cannot perform this +cleanup. A fallback or write failure can likewise leave cleanup for a later +successful attempt or the value TTL. + +The cleanup is not free. Every fenced write sends and temporarily stores the +complete serialized, possibly compressed payload before removing it. Include +that network transfer, Redis allocation, replication or AOF work, and stamp +round trip when estimating the load created by an oversized future buffer. + +### Shadow reads and fills + +[Shadow mode](shadow-validation.md) uses the same tracked protocol. A sampled +path can perform a tracked Redis read even when the remote serving ramp excludes +the key. A definitive `null` result can then attempt a tracked fill from the +caller-accepted source value, using the invocation's resolved remote TTL. + +An active future watermark rejects that fill and produces the bounded +`fill_blocked` shadow outcome. It does not reject or replace the value returned +to the caller. Caller-path request-local and process-local publication remains +independent when the remote serving layer is ramped out. + +The shadow read and fill are not atomic. An ordinary tracked cache write can +land between them, and either write can overwrite the other according to +arrival order when the watermark permits it. Shadow mode never repairs or +overwrites a non-null initial Redis payload; it only fills a definitive clean +miss. + +## Redis clock contract + +The bundled timestamp protocol assumes synchronized system clocks across every +Redis node eligible for primary promotion. + +Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache +does not detect or compensate for cross-node clock skew. If the assumption is +violated, failover can: + +- temporarily suppress tracked cache fills; or +- allow a pre-invalidation value to remain readable until it expires or a later + invalidation advances the watermark past its timestamp. + +Monitor and bound the maximum negative clock skew across all promotion-eligible +nodes. Include that bound when sizing `futureBufferMs`. + +## Watermark durability + +Watermarks are invalidation state, not disposable cache entries. Redis must +preserve each marker for its derived TTL with `noeviction` or an equivalent +guarantee. Choose persistence, restore, and failover behavior that matches the +application's consistency requirements. + +Losing a marker through eviction, failover, restore, or external deletion +removes its prior publication fence. A missing marker makes tracked reads miss, +but a later tracked write creates a new baseline and can publish data that a +lost future watermark would have rejected. + +Redis replication is asynchronous. DialCache does not issue `WAIT` and does not +provide strong consistency across failover. + +## Watermark lifetime + +Tracked writes create a missing baseline watermark and ensure its TTL is at +least the value TTL plus one minute. They never shorten a longer or persistent +watermark TTL. Because cache TTLs cap at 365 days, the derived marker TTL can +reach 365 days plus the fixed one-minute margin. + +Invalidation ensures the TTL covers both the requested future buffer and any +still-future existing watermark, plus one minute. It also preserves a longer or +persistent TTL. The one-minute safety margin is fixed; there is no separate +configurable or global retention floor, and reads do not extend watermark +lifetime. + +## Choosing `futureBufferMs` + +`futureBufferMs` must be a nonnegative safe integer no greater than +`31_536_000_000` milliseconds (365 days). The API default is zero, but zero +provides no stale-publication protection once Redis time advances. + +Larger values, negative values, fractions, non-finite values, and wrong-type +values are rejected with a `RangeError` before DialCache records metrics, logs, +checks whether Redis is configured, or calls the client. An invalid buffer +therefore takes precedence over the missing-Redis `TypeError` and has no +invalidation telemetry side effects. + +Every production invalidation should pass a named, application-owned nonzero +value based on measured or conservatively bounded timings. Size it to cover: + +- maximum expected negative clock skew between promotion-eligible Redis nodes; +- source visibility or replication lag; +- the full remaining tail of any fallback that may already have observed the + pre-mutation value; +- `serializer.dump`; +- synchronous compression or raw-payload escaping; +- Redis client queue and network latency for the full placeholder payload; +- the native placeholder `SET` and the tracked stamp script, including their + ordered dispatch and settlement; and +- a safety margin. + +Include the remaining lifetime of any sampled shadow fill based on a source +read that may have observed the pre-mutation state. A shadow deadline can stop +work before write dispatch, but it cannot prove that an already-dispatched +Redis command did not execute. + +Account for the underlying client's queue, dispatch, retry, and settlement +bounds as well as DialCache's shadow deadline. + +Invalidate only after the source mutation commits. + +Underestimating the interval can allow a delayed stale fallback to repopulate +Redis after the watermark window ends. Overestimating it lengthens the tracked +Redis miss and write-suppression window, increasing fallback load without +publishing stale values. + +A larger buffer does not delay or suppress returning fallback values to +callers. + +## Failure behavior and telemetry + +The bundled adapters dispatch invalidation with `EVALSHA` and retry a rejected +dispatch once with the script source through `EVAL`. Because its monotonic +update only advances the watermark and widens its lifetime, duplicate +execution after an ambiguous first result is safe. A successful recovery is +internal to the adapter and produces no DialCache error or retry metric. See +[Mutation retries and ambiguity](redis.md#mutation-retries-and-ambiguity) for +adapter-specific error handling. + +For a valid buffer, DialCache invokes the configured invalidation metric hook +with `layer="remote"` before it checks the Redis prerequisite. + +Missing configuration and Redis write failures then follow the same observable +failure path: DialCache logs `Error writing DialCache invalidation watermark`, +invokes the configured error metric hook with `useCase="watermark"`, +`layer="remote"`, `error="invalidation"`, and `inFallback=false`, then rethrows +the original error. Logger and metrics callback failures are isolated and +cannot replace that rejection. + +A surfaced mutation failure is ambiguous: Redis may have advanced the +watermark before the client lost the reply. Do not interpret the rejection as +proof that nothing executed. Repeating `invalidateRemote()` after the source +mutation has committed is safe and advances or preserves the fence, but may +extend the future miss window. + +## In-memory layers remain local + +Targeted invalidation is remote-only. `invalidateRemote` does not evict existing +request-local or process-local entries. + +Strongly invalidated mutable data should disable request-local and process-local +caching. A short process-local TTL is appropriate only when the application +explicitly accepts that bounded stale-read window. + +If those layers remain enabled, their existing values can be returned without +reaching the remote watermark. diff --git a/docs/maintainers.md b/docs/maintainers.md new file mode 100644 index 0000000..f67dd86 --- /dev/null +++ b/docs/maintainers.md @@ -0,0 +1,190 @@ +# Maintainer guide + +[Back to the README](../README.md) + +## Validation + +Use the repository's pinned pnpm version through Corepack: + +```bash +corepack pnpm install --frozen-lockfile +corepack pnpm check +corepack pnpm test:integration +``` + +`pnpm check` runs strict typechecking, the unit suite with coverage, +bundles/declarations, and packed ESM/CJS consumer tests. The integration suite +uses Testcontainers and requires a working Docker-compatible container runtime +for Redis, Valkey, and Redis Cluster. + +CI runs development and integration checks on Node.js 24, then switches to the +exact 22.x consumer floor, Node.js 22.15.0, to prove both the packed package and +`node:zlib` zstd support. The published engine range is +`>=22.15.0 <23.0.0 || >=23.8.0`, because Node.js 23.0 through 23.7 do not expose +the required zstd API. + +Keep the consumer floor separate from the development runtime so a dependency, +emitted syntax, or runtime API cannot silently raise the published +requirement. + +Before changing a compatibility-sensitive surface, identify and extend the +corresponding packed, unit, and integration assertions: + +- package root and explicit adapter/protocol entry points in packed ESM and CJS + consumers; +- full cache-key identity, encoding, namespace behavior, and Redis Cluster hash + tags; +- deterministic serving- and shadow-ramp assignment, whose independent cohorts + must not reshuffle across releases, plus nested shadow-policy snapshot, + overlay, validation, and legacy `shadowRamp` rejection; +- `coalesce` omission defaulting to enabled, sparse boolean overlays, explicit + opt-out in both request and process scopes, independent deadlines and writes, + settled request-local reuse, and malformed-value fail-open behavior; +- native `GET`/`MGET` reads, native untracked `SET` writes, and the ordered + tracked placeholder-`SET` plus stamp-script pair, including exact frame + encoders, script reply domains, the root-exported placeholder-loss error, + wrong-type behavior, Redis Cluster routing, and removed read/write-script + exports; +- tracked invalidation plus tracked and untracked shadow behavior, + mixed-version serializer behavior, and the ownership and immutability + contract for retained string and `Buffer` payloads; +- default-on zstd configuration and validation, binary envelope collisions, + decompression caps, raw fallback, mixed-version upgrades and rollbacks, + first-party and optional custom-adapter metrics, and the exact Node.js floor; +- rejection and bounded error telemetry when `invalidateRemote()` is called + without a configured Redis client; +- shadow confirmation, clean-miss fill, capacity, deadline, detached work, and + payload-release behavior, plus default-off confirmed-mismatch logging and its + byte-capped native-JSON detail fields; +- exhaustive public unions and packed exports, including `MetricLayer`, + `ShadowValidationOutcome`, `ShadowComparator`, `ShadowConfig`, + `CompressionConfig`, compression metric types, and Redis protocol error + classes; and +- bounded metrics names, labels, reasons, error categories, scopes, outcomes, + units, and observer isolation from synchronous throws and rejected thenables. + +When changing user-facing examples, parse TypeScript fences, validate local +files and anchors, verify that README repository links are absolute for npm +rendering, and inspect the packed README. The package ships `README.md` but not +`docs/`. + +See [Shadow validation](shadow-validation.md) for the contract that the shadow +unit, integration, adapter, package, and benchmark assertions protect. + +## Cache-path benchmark + +From a repository checkout, install dependencies and run: + +```bash +corepack pnpm benchmark:request-local +``` + +The command builds `dist` before reporting ten scenarios: + +- sequential request-local hits; +- sequential process-local hits; +- enabled bounded fallbacks; +- request-local coalescing fan-out; +- process coalescing fan-out; +- Redis read-deadline coalescing; +- tracked Redis hits with shadow omitted; +- tracked Redis hits outside a partial shadow cohort; +- ramped-down Redis shadow-read detachment; and +- ramped-down Redis shadow-fill detachment. + +The benchmark is a maintainer tool and is not included in the published +package. + +It asserts fallback counts, coalescing state, returned values, semantic Redis +calls, cleaned-up deadline timers, stable shadow exclusion, tracked dark reads +and fills, caller detachment, mismatch confirmation, and the absence of shadow +writes or invalidations on warm hits. It deliberately applies no timing +threshold. + +Override its work sizes with: + +- `DIALCACHE_BENCH_ITERATIONS`; and +- `DIALCACHE_BENCH_FANOUT`. + +## Redis write benchmark + +With a Redis server reachable at `REDIS_URL` (default +`redis://127.0.0.1:6379`), run: + +```bash +corepack pnpm benchmark:redis-write +``` + +The command builds `dist`, then measures eight sequential configurations: +tracked and untracked writes at 100 B, 10 KiB, 100 KiB, and 1 MiB. It reports +server-side command time per write from `INFO commandstats`, plus client-side +p50 and p95 latency. For tracked writes, the `EVALSHA` entry envelopes the +stamp script's internal command cost. + +This benchmark is a maintainer diagnostic and is not included in the published +package. It deliberately has no semantic assertion or timing threshold: +absolute results depend on the machine, Redis engine, payload, and ambient +load. + +Run it only against a dedicated disposable or development Redis. It writes +fixed `benchmark:write:*` keys and executes `CONFIG RESETSTAT` before every +sample, so the client needs that permission and the command erases the +server's accumulated command statistics. The script calls the semantic +adapter's `write()` method directly with prebuilt payloads; it measures neither +serializer nor compression cost. + +Compare implementations only with fresh alternating samples in the same +environment, and preserve correctness coverage in unit, packed-package, and +live integration tests. Scale every iteration count with +`DIALCACHE_BENCH_WRITE_SCALE`. + +## Releasing + +Publishing starts by manually running the `Release` workflow from current +`main`. + +After the package checks pass, Semantic Release selects the next version from +Conventional Commits since the highest stable `vX.Y.Z` tag: + +- while the package is pre-1.0, breaking changes bump minor and retain their + `BREAKING CHANGE:` footers for full release notes; +- `feat` bumps minor; and +- every other normal PR-title type bumps patch. + +Patch types are `fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, +`chore`, `ci`, and `revert`. The highest required bump wins. + +Major bumps resume when the project cuts 1.0.0. `release.config.mjs` implements +this policy; change it and this guide together so the documented release table +cannot drift from automation. + +The workflow opens a `release: ` pull request whose only change is the +matching `package.json` version. `release` is a reserved Conventional Commit +type configured not to request another release, so the version-control commit +does not cause an extra bump. + +GitHub marks workflow runs for a pull request opened with `GITHUB_TOKEN` as +approval-required. Approve those runs, review the pull request, and squash-merge +it normally through the protected branch. + +The merge triggers the publish job. Before any release side effect, it verifies: + +- current `main`; +- the release commit subject; +- the one-file diff; +- the package version; +- the absent tag; and +- Semantic Release's independently calculated version and commit. + +It then reruns the package checks and asks Semantic Release to: + +1. create the matching Git tag; +2. publish the public npm package with provenance; and +3. publish the GitHub release. + +The repository must enable **Allow GitHub Actions to create and approve pull +requests** under Actions workflow permissions. + +The workflow uses that capability only to create the version pull request. It +never approves or merges one, and no ruleset bypass actor or persistent release +credential is required. diff --git a/docs/observability.md b/docs/observability.md new file mode 100644 index 0000000..63f3744 --- /dev/null +++ b/docs/observability.md @@ -0,0 +1,448 @@ +# Observability + +[Back to the README](../README.md) + +Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the +constructor. `new DialCache()` does not import a metrics backend, register +collectors, or emit metrics. + +DialCache provides first-party adapters for Prometheus and Datadog. Both use +caller-created, caller-owned clients and preserve one backend-neutral set of +bounded labels. + +## Prometheus + +Install `prom-client` separately: + +```bash +npm install prom-client@^15.1.3 +``` + +Create the registry your application owns, then pass an explicit adapter to +DialCache: + +```ts +import { Registry } from "prom-client"; +import { DialCache } from "dialcache"; +import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; + +const registry = new Registry(); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createPrometheusDialCacheMetrics({ + registry, + prefix: "myapp_", + }), +}); + +app.get("/metrics", async (_req, res) => { + res.type(registry.contentType).send(await registry.metrics()); +}); +``` + +The adapter requires a caller-owned `Registry`. It never uses the global +default registry, and it does not clear or otherwise own the registry +lifecycle. + +Multiple adapters with the same registry and prefix reuse existing collectors +when their type, help, labels, histogram buckets, and exemplar mode match. +Adapter construction fails before registering anything if a same-name +collector has an incompatible schema. Use a unique prefix or separate registry +to resolve a collision. + +### Prometheus metrics + +The names below exclude the optional caller-selected prefix: + +| Metric | Type | Labels | Description | +| --- | --- | --- | --- | +| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by request-local or process scope | +| `dialcache_shadow_validation_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `outcome` | Terminal outcomes for sampled Redis shadow jobs | +| `dialcache_compression_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `outcome` | Bounded Redis payload compression and decompression outcomes | +| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes, before compression | +| `dialcache_stored_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Prepared Redis payload size in bytes, after compression and escaping | +| `dialcache_compression_ratio_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Compressed-to-original payload size ratio for compressed writes | +| `dialcache_compression_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Compression and decompression latency in seconds | + +The disabled reasons are: + +- `context`; +- `policy_disabled`; +- `invalid_ttl`; +- `invalid_ramp`; +- `ramped_down`; and +- `config_error`. + +`policy_disabled` means that a process-local or remote layer has no effective +TTL after runtime overlays. This is an intentional policy result, including the +default when `defaultConfig` is omitted, rather than a configuration-loading +failure. + +Every metric includes `cache_namespace`, even disabled-context, +key-construction, coalescing, and invalidation paths that do not have a +constructed key. Its value is `DialCacheConfig.namespace`, which defaults to +`urn`. + +The `layer` label is: + +- `request_local`; +- `local`, meaning process-local; +- `remote`; +- `remote_shadow` for Redis reads, fills, serialization, compression, and + payload sizes performed by detached shadow jobs; or +- `noop` for disabled-context, key-construction, and config-provider failures + where no cache layer was reached. + +The bounded `scope` label on `dialcache_coalesced_counter` distinguishes +`request_local` from `process`. `scope="process"` coordinates calls only within +one `DialCache` instance; separate instances in the same process do not share +in-flight state. A use case with `coalesce: false` emits no coalesced counter; +each caller instead emits its own request, miss, duration, and error metrics. + +### Shadow outcomes + +`dialcache_shadow_validation_counter` reports one terminal outcome for each +admitted or explicitly dropped shadow job. Datadog exposes the same bounded +outcomes through `dialcache.shadow.count`: + +| `outcome` | Meaning | +| --- | --- | +| `match` | The cached and source values matched. | +| `mismatch` | They differed, and a confirmation read found the original Redis payload unchanged. | +| `superseded` | They differed, but the Redis payload changed or disappeared before confirmation. | +| `filled` | A clean shadow miss was populated successfully. | +| `fill_blocked` | An invalidation watermark blocked a tracked clean-miss fill; compliant untracked writes do not produce it. | +| `fill_error` | Preparing the payload (serialization or compression) or writing a clean-miss fill failed. | +| `redis_error` | The initial detached Redis read failed. | +| `source_error` | The source-of-truth read failed. | +| `deserialization_error` | The retained Redis payload could not be deserialized for comparison. | +| `comparison_error` | The comparator threw or did not return a synchronous boolean. | +| `confirmation_error` | The confirmation Redis read failed. | +| `timeout` | The shadow deadline expired. | +| `dropped` | Per-key deduplication or the instance flight cap rejected the job. | + +The outcome counter deliberately has no `layer` or cache-id label. Operational +Redis metrics produced inside the same job use `layer="remote_shadow"`, which +keeps detached work separate from caller-serving `layer="remote"` telemetry. +See [Shadow validation](shadow-validation.md) for the read, confirmation, fill, +and deadline semantics behind these outcomes. + +### Compression metrics + +Compression telemetry is bounded and uses `layer="remote"` for caller-serving +work or `layer="remote_shadow"` for detached shadow work. + +Write-side outcomes are: + +- `compressed`: zstd plus its envelope was smaller and selected for the + prepared Redis payload; +- `below_threshold`: the serialized payload did not reach the configured + threshold; +- `not_smaller`: compression ran, but the marked result was not smaller than + the raw stored form; and +- `write_over_limit`: the serialized value exceeded the 512 MiB decompression + ceiling and was kept raw for the attempted write. This is a capacity signal, + not an error. + +Read-side outcomes are: + +- `decompressed`: a marked zstd payload was restored; +- `fallback_raw`: a marked payload was not valid zstd and was passed unchanged + to the serializer; and +- `read_over_limit`: decompression would exceed the 512 MiB ceiling, so the + stored bytes were passed unchanged to the serializer. Treat this as a + corruption or integrity signal. + +Raw reads do not emit a compression outcome. With `compression: false`, new +writes are still escaped when necessary but emit no compression outcome; reads +continue to report marked values because disabling writes does not disable +decoding. + +`dialcache_size_histogram` measures serializer output before compression and is +the distribution to use when selecting `thresholdBytes`. +`dialcache_stored_size_histogram` measures the prepared bytes after compression +or binary-envelope escaping. DialCache records it before the shadow deadline +gate and before calling the Redis client, so it is not proof that a write was +dispatched or succeeded. The ratio histogram is emitted when compression +selects the smaller representation, at the same pre-write stage. + +Compression duration is observed when zstd runs and produces either +`compressed` or `not_smaller`; decompression duration is observed for each +marked payload that produces a read-side outcome. + +A zstd exception while preparing a write records `error="compression"` and +the cache write fails open. Decompression rejects neither the cache call nor +the observer path directly: an unreadable payload reaches the configured +serializer, whose rejection follows the existing refreshable-miss path and +records `serialization_load`. + +zstd work is synchronous on the Node.js event loop. Use the duration, ratio, +and pre/post-size series together when changing the threshold or level; a good +space ratio does not make an event-loop stall acceptable. See +[Redis payload compression](redis.md#compression) for the envelope, limits, +and mixed-version rollout contract. + +### Confirmed mismatch warnings + +Shadow metrics remain bounded and contain no cache ids or values. A use case can +separately set `shadow.logMismatches: true` to emit one warning after a terminal +`mismatch` is confirmed. Logging is default-off, does not replace the outcome +metric, and does not activate shadow work without the `shadowValidation` hook. + +The warning contains stable metadata, the logical cache key capped at 2 KiB, +and independently generated native-JSON strings for the cached and source +comparator inputs capped at 8 KiB each. Those fields are value-bearing, and +truncation is not redaction. + +See +[Confirmed mismatch logging](shadow-validation.md#confirmed-mismatch-logging) +for confirmation semantics, exact fields, JSON behavior, operational limits, +and the required data-handling review. + +## Datadog + +Install `hot-shots` separately: + +```bash +npm install hot-shots@^17.0.0 +``` + +Create the DogStatsD client your application owns, then pass it to the Datadog +adapter: + +```ts +import StatsD from "hot-shots"; +import { DialCache } from "dialcache"; +import { createDatadogDialCacheMetrics } from "dialcache/datadog"; + +const dogStatsD = new StatsD({ + host: process.env.DD_AGENT_HOST ?? "127.0.0.1", + globalTags: { + service: "users-api", + env: process.env.DD_ENV ?? "development", + }, + errorHandler: (error) => + logger.warn("DogStatsD error", { error }), +}); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createDatadogDialCacheMetrics({ + client: dogStatsD, + observationMetricType: "distribution", + namespace: "dialcache", + }), +}); + +function shutdown(): void { + // Drain outstanding cache operations before application shutdown. + dogStatsD.close(); +} +``` + +`hot-shots` is the supported and tested client, but the adapter depends only on +the exported `DatadogDogStatsDClient` structural interface. + +DialCache does not: + +- import or install `hot-shots`; +- create a client; +- flush buffers; +- close sockets; or +- otherwise own the client lifecycle. + +### Distribution or histogram + +`observationMetricType` is required. + +Choose `"distribution"` when latency and size percentiles must aggregate across +hosts. Enable the desired distribution percentiles and aggregations in +Datadog. + +Choose `"histogram"` when host-level histogram aggregation matches the existing +Datadog setup. The choice applies uniformly to every duration, size, and ratio +observation emitted by the adapter. Both modes produce Datadog custom metrics. + +Distribution volume scales with unique tag-value combinations. Datadog counts +five baseline aggregations per combination; enabling percentile aggregations +adds five more. Review +[Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) +before rollout. + +Do not send both observation types under the same metric namespace. When +changing types, use a new namespace during migration so one metric identity +never mixes histogram and distribution points. + +### Datadog namespaces + +`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to +`dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache +namespace emitted as the `cache_namespace` tag. + +The Datadog metric namespace must: + +- start with a letter; +- contain only letters, numbers, underscores, and dot-separated non-empty + segments; and +- produce final metric names no longer than 200 characters. + +The adapter rejects invalid namespaces and overlong final names instead of +relying on client-side normalization. + +A `hot-shots` `prefix` is applied after the adapter constructs the name. Include +that prefix when checking final length, and avoid accidentally combining it +with the adapter namespace. Client-level `globalTags` are appended by +`hot-shots`; the table below lists only tags added by DialCache. + +### Datadog metrics + +The adapter emits exact increments of `1` for counters and preserves seconds +and bytes without unit conversion: + +| Metric | Type | Tags | Description | +| --- | --- | --- | --- | +| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | +| `dialcache.shadow.count` | Count | `cache_namespace`, `use_case`, `key_type`, `outcome` | Terminal outcomes for sampled Redis shadow jobs | +| `dialcache.compression.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `outcome` | Bounded Redis payload compression and decompression outcomes | +| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes, before compression | +| `dialcache.stored.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Prepared Redis payload size in bytes, after compression and escaping | +| `dialcache.compression.ratio` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Compressed-to-original payload size ratio for compressed writes | +| `dialcache.compression.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Compression and decompression latency in seconds | + +Client throws and rejected returned thenables are isolated by DialCache's +fire-and-forget observer boundary. Buffered transport failures that happen +after the client call returns remain outside that boundary. Configure the +DogStatsD client's error handling and shutdown behavior as part of application +ownership. + +## Error categories + +The `error` label reports the operation that failed instead of copying the +thrown value's class or `Error.name`: + +| `error` | Meaning | +| --- | --- | +| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | +| `config_resolution` | Runtime or layer configuration validation or resolution failed | +| `cache_read` | A process-local read or non-timeout remote read failed | +| `cache_read_timeout` | A remote read exceeded its effective DialCache deadline | +| `cache_write` | A process-local or remote cache write failed; native tracked writes include the observable lost-placeholder race described below | +| `serialization_load` | Deserializing a Redis payload failed | +| `serialization_dump` | Serializing a value for Redis failed | +| `compression` | zstd compression failed while preparing a Redis write | +| `invalidation` | Writing an invalidation watermark failed | +| `fallback` | The source loader failed or exceeded its DialCache deadline | +| `unknown` | Reserved for a future failure site that cannot be classified otherwise | + +These values are defined by the backend-neutral core and are identical for +every adapter. + +A valid `invalidateRemote()` call without a configured Redis client is still an +invalidation attempt: DialCache records `dialcache_invalidation_counter` (or +`dialcache.invalidation.count`), logs the failure, records +`error="invalidation"`, and rejects with the original focused `TypeError`. +Invalid `futureBufferMs` input is rejected before these observers run. + +Remote-read timeouts use `layer="remote"` and `in_fallback="false"`. They are +errors rather than misses, and the remote get-duration observation includes +the wait. Coalesced followers do not multiply the timeout error. Deadline +details remain out of labels and are available on the logged +`RedisReadTimeoutError`. + +A tracked native write first stores an unreadable placeholder and then stamps +that exact placeholder through the small mutation script. If another write +overwrites it, it expires, or a watermark-fenced write removes it before the +stamp, the adapter raises the root-exported +`DialCacheRedisPlaceholderLostError`. DialCache records one +`error="cache_write"`, suppresses publication of that write, and logs a warning. + +Same-key write contention can therefore create a benign, self-healing floor of +these errors around hot-key expiry. Keep the metric bounded, use the error +class in structured logs or direct adapter calls to distinguish the case, and +rate-limit the warning sink when that contention is expected. + +Raw thrown values, error names, messages, cache ids, arguments, and Redis keys +are never included in labels. When DialCache logs a cache-plumbing failure, the +raw details remain available through the configured logger; not every metric +error or shadow outcome has a matching log entry. + +The explicitly opted-in confirmed-mismatch warning is a separate value-bearing +log and does not alter the metric schema. + +`in_fallback` remains the explicit distinction between cache plumbing and +application fallback failures. + +## Custom adapters + +Implement `DialCacheMetricsAdapter` and pass it through +`new DialCache({ metrics })` for another telemetry backend. + +| Hook | Required | Value | +| --- | --- | --- | +| `request(labels)` | yes | One active cache-layer lookup. | +| `miss(labels)` | yes | One cache miss. | +| `disabled(labels)` | yes | One skipped layer or no-layer invocation with a bounded `reason`. | +| `error(labels)` | yes | One bounded failure site with `inFallback`. | +| `invalidation(labels)` | yes | One explicit remote invalidation call. | +| `coalesced(labels)` | no | One follower that joined request-local or process-scoped work. | +| `shadowValidation(labels)` | no | One terminal sampled-shadow outcome. This hook must be implemented for shadow jobs to execute. | +| `compression(labels)` | no | One bounded compression or decompression outcome. | +| `observeGet(labels, seconds)` | yes | Cache-read duration in seconds. | +| `observeFallback(labels, seconds)` | yes | Fallback duration in seconds. | +| `observeSerialization(labels, seconds)` | yes | Serializer dump/load duration in seconds. | +| `observeSize(labels, bytes)` | yes | Serialized remote payload size in bytes, before compression. | +| `observeStoredSize(labels, bytes)` | no | Prepared remote payload size in bytes, after compression and escaping; emitted before client dispatch. | +| `observeCompressionRatio(labels, ratio)` | no | Compressed-to-original size ratio when compression selects the prepared representation. | +| `observeCompression(labels, seconds)` | no | Compression or decompression duration with `operation="compress"` or `operation="decompress"`. | + +The root package exports `DialCacheMetricsAdapter` and every associated label, +reason, error-kind, layer, scope, and shadow-outcome type, including +`ShadowValidationMetricLabels`, `ShadowValidationOutcome`, +`CompressionMetricLabels`, `CompressionOperationMetricLabels`, and +`CompressionOutcome`. +`shadowValidation` remains optional so existing custom adapters keep +compiling, but DialCache does not admit shadow work when the configured +adapter omits it. The Prometheus and Datadog adapters implement the hook. + +The compression hooks are also optional for source compatibility with existing +custom adapters. They control observation only: omitting them does not disable +compression or decompression. The Prometheus and Datadog adapters implement +all four hooks. + +Metrics and logger methods are typed `void` and invoked as fire-and-forget +observers. DialCache also defensively consumes, but never awaits, a thenable +returned at runtime. + +Synchronous throws and asynchronous rejections are isolated so telemetry +cannot change cache correctness, fallback results, or shadow outcomes. + +A custom adapter may buffer or transmit asynchronously, but it owns delivery, +flushing, resources, and shutdown after the call returns. Keep +application-owned namespace, use-case, and key-type labels stable and +low-cardinality, and preserve the seconds and bytes units shown above. + +Every backend-neutral label object exposes the logical namespace as camel-case +`cacheNamespace`. Map it to the backend's `cache_namespace` label or tag. This +field is present even when no key or cache layer was reached. + +Omit `metrics` to disable metrics entirely. Because shadow jobs require an +observable terminal outcome, omitting metrics also disables shadow execution +even when a key policy sets `shadow.ramp` or enables +`shadow.logMismatches`. diff --git a/docs/redis.md b/docs/redis.md new file mode 100644 index 0000000..4b79810 --- /dev/null +++ b/docs/redis.md @@ -0,0 +1,742 @@ +# Redis and Valkey + +[Back to the README](../README.md) + +DialCache's remote TTL layer supports standalone Redis, standalone Valkey, and +Redis Cluster. The application creates, connects, configures, drains, and closes +the underlying client. DialCache borrows a client-independent +`DialCacheRedisClient` adapter and does not own the connection lifecycle. + +Sampled non-serving Redis reads and fills use the same adapter and preserve the +operation's tracked or untracked mode. See +[Redis shadow validation](shadow-validation.md) for eligibility, comparison, +capacity, metrics, and rollout behavior. + +## Install a client + +Choose one supported integration: + +```bash +# node-redis +npm install redis@~4.7.1 + +# or Valkey GLIDE +npm install @valkey/valkey-glide@^2.0.0 +``` + +## node-redis + +Register DialCache's two mutation scripts when creating the client, connect +it, and pass the DialCache-compatible adapter to `DialCache`: + +```ts +import { createClient } from "redis"; +import { DialCache } from "dialcache"; +import { + createNodeRedisDialCacheClient, + dialcacheRedisScripts, +} from "dialcache/node-redis"; + +const redisClient = createClient({ + url: process.env.REDIS_URL ?? "redis://127.0.0.1:6379", + scripts: dialcacheRedisScripts, + disableOfflineQueue: true, + commandsQueueMaxLength: 1_000, + socket: { connectTimeout: 2_000 }, +}); + +await redisClient.connect(); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { + client: createNodeRedisDialCacheClient(redisClient), + }, +}); + +async function shutdown(): Promise { + // Stop new work and await every cached call and invalidation first. + await redisClient.quit(); +} +``` + +`redis.client` is required when the remote layer is configured. Node-redis +users should register the supplied scripts and wrap the connected client with +`createNodeRedisDialCacheClient` as shown above. Active remote reads have a +50-millisecond DialCache deadline by default. Set `redis.readTimeoutMs` for an +instance-wide value or use `DialCacheKeyConfig.remoteReadTimeoutMs` for +per-use-case static and runtime policy. + +Use node-redis's promise-mode client; `legacyMode` is not supported. Treat +`dialcacheRedisScripts` as adapter wiring rather than a direct write API. Its +`dialcacheWriteTrackedStamp` method returns the raw `0 | 1 | 2` script reply; +direct callers must pass that reply through `resolveTrackedRedisWriteReply` +from `dialcache/redis-protocol` so reply `2` becomes a lost-placeholder error. + +Local-only caching does not require a Redis client, but the explicit remote +maintenance operation `invalidateRemote()` does. It rejects when Redis is not +configured so a caller cannot mistake an absent watermark write for successful +invalidation. See [Targeted invalidation](invalidation.md) for the complete +contract. + +The registered scripts stamp tracked writes and advance invalidation +watermarks. Reads and untracked writes use native Redis commands. Node-redis +performs its normal script-cache recovery for the stamp script; the adapter's +additional invalidation recovery is described under +[Mutation retries and ambiguity](#mutation-retries-and-ambiguity). + +Deployments using tracked invalidation must also satisfy the +[watermark durability](invalidation.md#watermark-durability) contract. + +## Valkey GLIDE + +Pass an already-created standalone or cluster client and the exact module +namespace that created it: + +```ts +import * as valkeyGlide from "@valkey/valkey-glide"; +import { DialCache } from "dialcache"; +import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; + +const glideClient = await valkeyGlide.GlideClient.createClient({ + addresses: [{ host: "127.0.0.1", port: 6379 }], + requestTimeout: 2_000, + advancedConfiguration: { + connectionTimeout: 2_000, + }, +}); + +const redisClient = createValkeyGlideDialCacheClient( + glideClient, + valkeyGlide, +); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: redisClient }, +}); + +function shutdown(): void { + // Drain cached calls and invalidations before releasing resources. + glideClient.close(); +} +``` + +DialCache uses the supplied namespace's `Batch`, `ClusterBatch`, client +constructors, and `Decoder.Bytes` value without importing a GLIDE runtime +itself. Passing the same module namespace that created the client prevents +linked workspaces or applications with another installed GLIDE version from +mixing native objects. + +Pass a direct `GlideClient` or `GlideClusterClient` instance. Wrappers should +implement `DialCacheRedisClient` directly. The returned adapter is stateless, +owns no script handles, and needs no disposal; the application closes the +underlying GLIDE client after its work drains. + +## Bundled Redis operations + +The node-redis and GLIDE adapters preserve the same semantic protocol while +using each client's native command and routing APIs. + +### Reads + +- An untracked read is one native `GET`. +- A tracked read is one atomic `MGET valueKey watermarkKey`. Cluster adapters + explicitly route it to the slot primary, even when replica reads are enabled, + so replica lag cannot hide an invalidation watermark. +- Missing, short, unsupported-version, or placeholder frames are clean misses. + A tracked read also misses when the watermark is missing or malformed, or + when `createdAt <= watermark`. + +The value and watermark must share a Redis Cluster slot; DialCache's generated +tracked keys do. Redis returns the complete value before the adapter compares +its timestamp with the watermark. A large invalidated value can therefore use +network bandwidth on every attempted read until a successful fallback write +removes it or its TTL expires. See +[Invalidated payload transfer and cleanup](invalidation.md#invalidated-payload-transfer-and-cleanup). + +GLIDE standalone sends tracked `MGET` through a one-command non-atomic batch +so the client routes it to the primary instead of applying its ordinary +one-key read preference. `MGET` itself remains the single atomic snapshot; the +batch is deliberately non-transactional and does not consume caller-owned +`WATCH` state. + +Node-redis standalone sends `MGET` to the endpoint the application configured; +the adapter cannot discover or reroute a standalone replica connection. Point +that client at the authoritative primary when relying on tracked invalidation. + +DialCache keys must remain application-owned strings. Native Redis type rules +are intentionally visible: `GET` rejects a wrong-type untracked value, while +`MGET` returns a missing member for a wrong-type tracked value or watermark. +A wrong-type tracked value can be replaced by the fallback write when its +watermark is valid. + +A wrong-type or malformed watermark makes reads miss, then causes the stamp to +fail after the placeholder `SET`; repeated calls therefore fail open and reload +until that watermark state is repaired. A valid-version frame with an +unsupported payload encoding is a typed payload error rather than a miss. + +### Writes + +An untracked write is one native command: + +```text +SET valueKey frame PX cacheTtlMs +``` + +Its frame carries an informational client-clock timestamp. Untracked reads do +not consult that timestamp. + +A tracked write uses two commands, ordered on one connection without +`MULTI`/`EXEC`: + +1. `SET` writes the complete serialized payload in an unreadable version-0 + placeholder with a fresh nonce and the value TTL. +2. `WRITE_TRACKED_STAMP_SCRIPT` verifies that exact nonce, reads Redis time and + the watermark, and either promotes the placeholder to a readable frame, + unlinks it when the watermark fence is active, or reports that the + placeholder was lost. + +The nonce prevents a delayed stamp from publishing another writer's value. +The placeholder is a deliberate fail-safe: an interleaved or failed stamp is a +miss, not an unstamped cache hit. Because its `SET` replaces the prior frame, a +tracked write can briefly make a previously readable key miss while the stamp +settles. + +The pair is non-transactional so it does not consume caller-owned Redis +`WATCH` state. The bundled adapters enqueue or batch the pair in order. A +watermark fence returns `false`; DialCache returns the fallback value and does +not publish it process-locally. + +A missing, overwritten, or expired placeholder throws +`DialCacheRedisPlaceholderLostError`. Ordinary cached calls absorb that error +through the fail-open cache-write path, so same-key write contention can +produce benign bounded `cache_write` errors on hot keys. + +The adapter reports a failed `SET` as the write outcome even if the stamp also +settled. Because transport failures can be ambiguous, the `SET` may have +landed and the stamp may have promoted it despite the reported error. Never use +a cache-write rejection as proof that Redis was not mutated. + +### Mutation retries and ambiguity + +The bundled adapters dispatch invalidation with `EVALSHA`. If dispatch rejects, +they retry once with the monotonic, replay-safe script source through `EVAL`; +this also repairs a flushed script cache. A reply-domain violation is a +protocol error, not a retryable dispatch failure. If recovery fails, the retry +error surfaces. + +GLIDE attaches the original rejection as its `cause` when safe; node-redis does +not mutate the shared error objects it can use for disconnect failures. + +For the tracked stamp, node-redis uses its registered script's normal +`NOSCRIPT` recovery. GLIDE retries the stamp with `EVAL` only on `NOSCRIPT`, +because any other error may be an ambiguous result from a stamp that already +executed. The first tracked GLIDE write after a script-cache flush can +therefore pay one extra round trip. + +The retry is below the `DialCacheRedisClient` boundary. A successful recovery +is therefore not a DialCache error or retry metric, although Redis command +statistics can reveal the additional `EVAL`. + +Like any network mutation, a rejected write or invalidation can have executed +before the client reports failure. Do not add an outer `Promise.race` and +assume rejection proves non-execution; use finite client-native queue, +reconnect, dispatch, and response budgets. + +### Redis compatibility and ACLs + +The tracked stamp uses `UNLINK`, so the bundled protocol requires Redis 4 or a +compatible Valkey release. Redis Cluster deployments must allow multi-key +operations for keys in the same slot. + +At minimum, allow the client commands `GET`, `MGET`, `SET`, `EVALSHA`, and +`EVAL`. The scripts also invoke `TIME`, `GET`, `SET`, and `PTTL`; the tracked +stamp additionally invokes `PEXPIRE`, `UNLINK`, `GETRANGE`, and `SETRANGE`. +The bundled adapters do not require `SCRIPT LOAD`. + +Verify ACLs and proxy behavior before upgrading. A persistent stamp failure +still lets each placeholder `SET` replace the last readable value, while the +failed write suppresses process-local publication. Within one value-TTL +horizon, affected tracked keys can send all traffic to the source. Each lost +placeholder on a DialCache request path also records a bounded `cache_write` +error and emits a warning; size alerts and logger rate limits for expected +same-key contention. + +## Lifecycle ownership + +The application owns the complete Redis lifecycle: + +1. Create and connect the underlying client. +2. Construct the DialCache-compatible adapter. +3. Pass that adapter as `redis.client`. +4. During shutdown, stop starting DialCache-backed work. +5. Await every outstanding cached-function, `getOrLoad()`, and + `invalidateRemote()` promise, including fallbacks that may still write + Redis. +6. Drain or terminate client-native Redis work that may have outlived + DialCache's caller-serving or shadow-read wait. +7. Close the underlying connection. + +DialCache has no `close()` or drain method. It never disposes or closes caller +resources. + +Awaiting public DialCache promises does not drain detached shadow work. Shadow +scheduling and deadline timers are unreferenced, and there is no shadow drain +handle. + +A source read, serializer, Redis command, or metrics delivery started by a +shadow job can remain active during teardown. Stop new work before closing +dependencies and use their native drain or termination controls. An +already-dispatched shadow fill may have executed even if its outcome is lost. + +Both bundled adapters are resource-free views over caller-owned clients. They +have no `dispose()` method and own no connection, batch, or script handle. +After DialCache and detached client work drain, close the underlying node-redis +or GLIDE client directly. A DialCache read timeout does not prove that the +client-side invocation has settled. + +## Remote-read deadlines and async liveness + +Every caller-serving Redis read and each detached shadow read has a finite +monotonic deadline. DialCache uses this precedence for `cached()` and +`getOrLoad()`: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +Each explicit value must be a positive safe integer no greater than +2,147,483,647 milliseconds. Remote reads have no unbounded escape hatch. + +Outside an enabled scope and on an earlier in-memory hit, DialCache creates no +remote-read timer. A key ramped out of Redis serving creates no caller-serving +timer, but an independently selected shadow job can create unreferenced timers +for its same-mode `C0` and optional `C1` reads. + +### Caller-serving timeout and fail-open behavior + +When the deadline expires, DialCache: + +1. aborts the optional adapter signal; +2. logs a root-exported `RedisReadTimeoutError` carrying `useCase` and + `timeoutMs`; +3. records `cache_read_timeout`; +4. consumes and ignores any late read fulfillment or rejection; and +5. runs the source fallback. + +The deadline bounds caller wait and cache publication. It does not guarantee +server-side cancellation, and an event-loop-blocking operation cannot be +preempted. When control returns, DialCache still checks the monotonic deadline +before accepting the result. + +A remote read rejection or timeout does not count as a miss and never triggers +a second Redis operation. After fallback, an untracked key may still populate +an active process-local cache. A tracked key suppresses process-local +publication because watermark safety was not established. Request-local +memoization remains unconditional. + +Same-key callers in one request-local or process coalescing scope share the +leader's read, timer, and remaining budget. With `coalesce: false`, each caller +gets a full independent read budget and can start another remote read while a +prior client operation is still settling. + +The `fallbackTimeoutMs` timer is separate and starts only if and when the source +loader begins. The remote-read timer covers neither config resolution, +serializer loading, the fallback, Redis writes, nor invalidation. + +### Shadow-read deadlines + +Shadow `C0` and `C1` reads use the same effective `remoteReadTimeoutMs` and +cooperative abort signal as caller-serving reads. Their timers are +unreferenced. A `C0` failure or read timeout produces `redis_error`; the same +failure at `C1` produces `confirmation_error`. Both paths also record the +ordinary bounded Redis error under `layer="remote_shadow"`. + +The read deadline bounds DialCache's wait, not the underlying client +operation. Shadow capacity remains occupied while a timed-out raw Redis read +is still settling. The whole shadow job has a separate deadline described in +[Redis shadow validation](shadow-validation.md#capacity-deadlines-and-detachment). + +### Custom-client contract + +Custom adapters implement the complete client-independent read, write, and +invalidate contract: + +```ts +import type { + RedisCachePayload, + RedisInvalidationRequest, + RedisReadContext, + RedisReadRequest, + RedisWriteRequest, +} from "dialcache"; + +type Awaitable = T | Promise; + +interface DialCacheRedisClientContract { + read( + request: RedisReadRequest, + context?: RedisReadContext, + ): Awaitable; + write(request: RedisWriteRequest): Awaitable; + invalidate(request: RedisInvalidationRequest): Awaitable; +} +``` + +#### `read` + +- Decode native bulk-string replies with `decodeRedisFrame` or + `decodeTrackedRedisFrame` from `dialcache/redis-protocol`, or preserve their + exact behavior, and return the operation-owned serialized `string` or + `Buffer`, or `null` for a miss. +- Keep the payload stable after settlement because DialCache can retain it for + shadow work. An adapter that recycles response storage must return a + dedicated `Buffer`. +- For a tracked request, obtain the value and watermark atomically from one + authoritative primary snapshot. A missing or malformed watermark, or a + value at or behind it, is a miss. + +#### `write` + +- Normalize `cacheTtlMs` with `ceilSupportedCacheTtlMs`; positive fractional + milliseconds round up, and the result may not exceed `31_536_000_000` + milliseconds (365 days). +- Use `encodeRedisFrame` for the one-command untracked path. For a tracked + request, preserve the exact placeholder-and-stamp behavior described above + with `encodeTrackedRedisPlaceholder`, `WRITE_TRACKED_STAMP_SCRIPT`, and + `resolveTrackedRedisWriteReply`. +- On a non-fenced tracked write, create a missing baseline watermark and retain + it for at least the value TTL plus one minute without shortening a longer or + persistent lifetime. +- Return `true` only when the value was published and `false` only when the + watermark fence rejected it. Surface a lost placeholder as + `DialCacheRedisPlaceholderLostError`, not as `false`. + +#### `invalidate` + +- Accept `futureBufferMs` as a nonnegative integer no greater than + `31_536_000_000` milliseconds (365 days). +- Advance `watermarkKey` monotonically to at least server time plus that buffer. +- Retain the watermark long enough to cover the buffer and any still-future + existing watermark, plus one minute, without shortening a longer or + persistent lifetime. +- Reject on failure. + +`write()` returning `false` is a safe publication refusal, not an adapter error. +DialCache still returns the fallback value but skips the corresponding +process-local population. A thrown cache-write error fails open; a thrown +explicit invalidation error is rethrown to the caller. + +The optional `RedisReadContext` keeps existing one-argument readers +structurally compatible. Adapters should use its signal for cooperative +cancellation where their client supports it, but the core deadline remains +authoritative when they do not. + +The bundled node-redis adapter forwards the signal in per-command options. This +can remove queued work where supported, but aborting after dispatch cannot +unsend a command or prove that Redis stopped executing it. + +The GLIDE command API has no per-invocation signal, so a timed-out command may +continue inside the adapter. Its configured +[`requestTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.BaseClientConfiguration.html) +and +[`advancedConfiguration.connectionTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.AdvancedBaseClientConfiguration.html) +still bound client-native work. + +### Native operation budgets + +DialCache's read deadline bounds its caller wait, not the complete lifetime of +the underlying client work. Configure finite client-native budgets for: + +- connection establishment; +- reconnection and retries; +- offline queueing; +- dispatch; and +- response time. + +For node-redis 4.7, `socket.connectTimeout`, `disableOfflineQueue`, and +`commandsQueueMaxLength` bound connection or queue behavior but do not impose a +strict response deadline after dispatch. Use client-native shutdown or +termination behavior that matches the application's resource and ambiguity +requirements. + +Redis writes and invalidations, asynchronous `cacheConfigProvider` work, and +custom `Serializer` methods still require their own finite budgets. The same +is true for detached source, Redis, serializer, and telemetry work admitted by +shadow validation. + +Do not put writes or invalidations behind a bare `Promise.race`: rejecting the +outer promise neither removes queued work nor proves that a dispatched +mutation did not execute. + +DialCache's [fallback deadline](coalescing.md#fallback-deadlines) covers only the +source loader. Prefer resource-native budgets and cooperative cancellation for +every injected operation. + +## Serialization + +DialCache uses `JsonSerializer` by default. A cache operation can select a +typed serializer, and `redis.serializer` supplies the instance default when an +operation does not select one. Serializers run only for remote reads and +writes; request-local and process-local values remain native references. + +### Default JSON behavior + +DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no +runtime validation pass, so the default adds no traversal beyond JSON +serialization itself. A top-level `undefined` result is supported with an +internal sentinel. + +When `serializer.load` rejects a Redis payload, DialCache: + +1. records a `serialization_load` error; +2. counts the read as a remote miss; +3. runs the fallback; and +4. attempts to replace the rejected payload. + +A validating custom serializer can therefore treat an incompatible cached +value as a refreshable miss without adding a schema version to the cache key. +This replacement behavior describes the caller-serving path. Shadow work +reports `deserialization_error` for a non-null payload and never repairs it. + +`JsonSerializer` validates JSON syntax only. It cannot detect that a +structurally valid payload came from an incompatible application value schema. +Applications that retain one `useCase` across deployments must keep +default-JSON values backward compatible. + +For an incompatible change, either: + +- provide a serializer whose `load` method validates and rejects the old shape; + or +- change `useCase` to isolate the new cache entries. + +During a mixed deployment, mutually incompatible validating serializers can +repeatedly reject and replace each other's values. Correctness is preserved, +but expect additional fallback and Redis-write load until the rollout +converges. + +### Typed serializer requirement + +When a cached function or inline loader's resolved return type is statically +JSON-compatible, `serializer` is optional. This includes JSON primitives, +arrays, plain object or interface shapes, optional object fields, and a +top-level `undefined`. + +Types known not to survive the default round trip require a typed +`Serializer`: + +```ts +import { DialCache, type Serializer } from "dialcache"; + +const dialcache = new DialCache(); + +const dateSerializer: Serializer = { + dump: (value) => value.toISOString(), + load: (value) => + new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), +}; + +const getUpdatedAt = dialcache.cached( + (userId: string) => db.fetchUpdatedAt(userId), + { + keyType: "user_id", + useCase: "GetUpdatedAt", + cacheKey: (userId) => userId, + serializer: dateSerializer, + }, +); +``` + +The compile-time guard rejects known incompatible shapes such as: + +- `Date`, `Map`, and `Set`; +- `bigint`, symbols, and functions; +- Buffers and typed arrays; +- method-bearing class instances; +- required nested `undefined`; and +- `unknown` and `any`. + +The guard applies to every `cached()` declaration and `getOrLoad()` invocation +because active layers are selected at runtime. A global Redis serializer is not +parameterized by each returned type, so it cannot discharge this requirement. +Non-JSON operations must select a typed serializer. + +This guard is deliberately conservative rather than a proof of runtime data. +TypeScript cannot detect non-finite numbers, cyclic or shared references, +runtime getters, `toJSON` behavior, or data-only class instances that resemble +plain objects. Opaque, generic, or deeply recursive types may also require an +explicit serializer. + +Providing `Serializer`, including an explicitly typed +`JsonSerializer`, is a trusted caller assertion. DialCache does not perform +an additional serialize-and-deserialize cycle to validate it. + +Opted-in confirmed-mismatch logging is separate from cache serialization. It +uses native `JSON.stringify` on the deserialized cached snapshot and source +value and does not call the configured serializer again. Its byte caps are not +redaction; review the data-handling contract in +[Redis shadow validation](shadow-validation.md) before enabling it. + +Shadow validation can call `load` again for the same served payload and can +call `dump` after a ramped-down caller has received its source result. Custom +serializers must treat payloads and values as borrowed and immutable, return +independent values from repeated loads, and copy a Buffer before mutating it. +See [Data ownership and custom integrations](shadow-validation.md#data-ownership-and-custom-integrations). + +## Compression + +Redis payload compression is enabled by default and configured once per +`DialCache` instance: + +```ts +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + compression: { + thresholdBytes: 4_096, + level: 3, + }, + }, +}); +``` + +`thresholdBytes` must be a positive safe integer and defaults to 4,096 bytes. +`level` must be an integer from 1 through 22 and defaults to 3. Pass +`compression: false` to disable compression on future writes. This is an +instance-level write policy, not a per-use-case runtime ramp. + +DialCache measures the serializer output in bytes, then compresses it with +zstd only when it meets the threshold and the marked compressed form is +smaller than the raw stored form. Compression and decompression are +synchronous and run on the Node.js event loop. Benchmark representative value +sizes and zstd levels under production-like concurrency before lowering the +threshold or raising the level. + +Default-on compression requires zstd-capable `node:zlib`; DialCache validates +that support during construction. The package's supported Node.js range starts +at 22.15.0 in the 22.x line and excludes 23.0 through 23.7. The exact published +engine range is `>=22.15.0 <23.0.0 || >=23.8.0`. + +Reads always decode marked payloads, even when write-side compression is +disabled. That makes `compression: false` a safe way to stop producing new +compressed entries without orphaning existing ones. Binary serializer output +whose first byte is `0x00`, `0x01`, or `0x02` is also escaped on every write, +including when compression is disabled, so current readers can distinguish it +from the compression envelope exactly. + +### Size limits and failure behavior + +DialCache caps decompressed output at 512 MiB, matching Redis's value limit. +Serializer output above that cap is left raw for the Redis write rather than +compressed. + +If a marked value cannot be decompressed or would exceed the cap, DialCache +passes the original marked bytes to `serializer.load`; a validating serializer +will normally reject it and trigger the existing self-healing miss path. A +permissive custom serializer can instead accept those bytes, so monitor +`fallback_raw` and `read_over_limit` as payload-integrity signals rather than +assuming they always become misses. + +A write-side zstd exception records `error="compression"`, skips the Redis +write, and follows DialCache's fail-open cache-write path. The fallback result +still returns. Decompression outcomes are reported before serializer loading; +if loading then fails, the same read can also record `serialization_load`. + +Compression-aware metrics adapters can implement the optional `compression`, +`observeStoredSize`, `observeCompressionRatio`, and `observeCompression` +hooks. `observeSize` remains the serializer-output size before compression or +escaping; `observeStoredSize` measures the prepared payload afterward, before +the shadow deadline gate and Redis write. It does not prove that a write was +dispatched or succeeded. See +[Observability](observability.md#compression-metrics) for bounded outcomes and +the bundled Prometheus and Datadog metric names. + +### Rolling deployments and binary serializers + +Current readers accept frames from older releases. Older readers, however, do +not understand newly compressed or escaped payloads and will usually reject +them during deserialization, causing temporary fallback and refill churn in a +mixed deployment. For string and JSON serializers, a low-noise rollout is: + +1. deploy the new release everywhere with `compression: false`; +2. allow old readers to drain; and +3. enable compression in a later rollout. + +Apply the same consideration when rolling back while compressed entries still +exist. + +Before the escape envelope existed, arbitrary binary output could already +begin with an envelope marker. A legacy payload beginning with `0x00` followed +by `0x00`–`0x02`, or with `0x01`/`0x02` followed by a valid zstd stream, can be +misinterpreted by a current reader until it expires. If a custom serializer +can emit those prefixes, change the operation's `useCase` or other key-version +component for the migration. + +## Advanced wire protocol + +The core Redis boundary is the client-independent `DialCacheRedisClient` +interface. It exchanges serialized values as `string | Buffer` and does not +expose client-specific commands or wire encodings. + +The `dialcache/redis-protocol` entry point exports the exact bundled protocol +building blocks: + +- `decodeRedisFrame` and `decodeTrackedRedisFrame` for native read replies; +- `encodeRedisFrame`, `encodeTrackedRedisPlaceholder`, and the + `TrackedRedisPlaceholder` type for native writes; +- `ceilSupportedCacheTtlMs` for adapter-level write TTLs; +- `WRITE_TRACKED_STAMP_SCRIPT` and `INVALIDATE_CACHE_SCRIPT`; +- `resolveTrackedRedisWriteReply`; and +- `validateRedisSetReply` and `validateRedisScriptInvalidationReply`. + +The payload bytes inside the Redis frame are opaque to this adapter-level +protocol. Compression and escaping sit above it in DialCache core. Custom +adapters must preserve those bytes exactly and must not decompress or rewrite +them. + +Custom adapters can throw these root-exported error classes: + +- `DialCacheRedisPayloadError`; +- `DialCacheRedisPayloadEncodingError`; +- `DialCacheRedisProtocolError`; and +- `DialCacheRedisPlaceholderLostError`. + +They distinguish invalid runtime payload or reply shapes, unsupported +encodings, and lost tracked placeholders. DialCache records bounded +`cache_read`, `cache_read_timeout`, `cache_write`, or `invalidation` metrics by +failure site. + +Shadow validation adds no Redis protocol operation. It composes the ordinary +request shapes for the operation: `C0`, `C1`, and any clean-miss fill remain +tracked or untracked together. + +### Binary frame + +Redis values use a compact binary frame: + +```text +byte 1 format version +bytes 2-9 creation timestamp or placeholder nonce (eight-byte region) +byte 10 payload encoding (0 = UTF-8, 1 = raw binary) +bytes 11... opaque post-serialization payload +``` + +Version 1 is readable. A tracked placeholder uses version 0 and stores its +eight-byte nonce in the stamp region, so neither read path can serve it. The +stamp script promotes only its matching placeholder by replacing version and +nonce with version 1 and Redis time. + +An untracked frame is version 1 from the start and carries an informational +client-clock timestamp that untracked reads ignore. Redis TTL is authoritative, +so expiry metadata is not duplicated in the frame. + +The payload region contains the serializer output after any compression or raw +binary escaping. The frame encoding preserves whether that region is a string +or `Buffer`; strings use UTF-8 and Buffers need no base64 expansion. After the +adapter decodes the frame, DialCache interprets the optional compression +envelope and restores the serializer's representation before calling +`serializer.load`. diff --git a/docs/shadow-validation.md b/docs/shadow-validation.md new file mode 100644 index 0000000..302ee01 --- /dev/null +++ b/docs/shadow-validation.md @@ -0,0 +1,533 @@ +# Redis shadow validation + +[Back to the README](../README.md) + +Shadow validation lets a service exercise and inspect Redis behavior without +letting the shadow path choose the caller's result. It is useful for validating +warm entries against the source of truth and for bootstrapping clean misses +before increasing the Redis serving ramp. + +Shadow mode is an operational rollout tool, not a new serving layer or a +correctness boundary. It adds source and Redis work, provides best-effort +evidence through bounded metrics, and preserves each key's ordinary tracked or +untracked Redis mode. It relies on the same serialization, deadline, consistency, +and client-lifecycle contracts as the remote layer. + +## At a glance + +| Path reached by the caller | Caller receives | Selected shadow work | +| --- | --- | --- | +| Serving Redis hit, tracked or untracked | The decoded Redis value | Compare a later source read with the retained payload, then confirm a mismatch candidate with one same-mode Redis read. | +| Valid remote policy, but ramped out of Redis serving | The normal source result | Read Redis later without serving it. Compare a hit, or fill a clean miss from the accepted source result. | +| Serving Redis miss | The normal source result | None. The ordinary request path already performs fallback and fill. | +| Request-local or process-local hit | The in-memory value | None. Normal traversal never reached Redis. | + +Redis serving and shadow selection use independent deterministic cohorts. A +key can therefore be served without being shadowed, shadowed without being +served, selected for both, or selected for neither. + +## Configure a shadow cohort + +Shadow validation requires a valid remote TTL, a metrics adapter with the +optional shadow hook, and a positive `shadow.ramp`. Invalidation tracking is +optional and selects the Redis consistency mode rather than shadow eligibility: + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: dialCacheRedisClient }, + metrics, + // Per-instance cap; the default is 1. + shadowMaxInFlight: 4, +}); + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + // Optional for shadowing; adds watermark fencing to Redis reads and fills. + trackForInvalidation: true, + shadowComparator: (cached, source) => + cached.id === source.id && cached.version === source.version, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.REMOTE]: 300 }, + // Exercise and fill Redis without allowing it to serve this cohort. + ramp: { [CacheLayer.REMOTE]: 0 }, + shadow: { ramp: 5 }, + }), + }, +); +``` + +`shadow.ramp` accepts percentages from `0` through `100`. An effective value of +`0`, or omission from both baseline and runtime policy, disables shadow work; +`100` selects every otherwise eligible exact cache key. + +A sparse runtime overlay that omits the field inherits the baseline. +Intermediate values select a stable key cohort across calls and instances. The +shadow bucket is independent of the remote serving bucket, so equal ramp +percentages do not select the same keys. + +Use `shadow: { ramp: 100 }` when every eligible invocation should exercise the +non-serving Redis path before a later serving-ramp increase. This still +observes only keys invoked during the shadow period; it is not a full keyspace +scan or warming guarantee. + +The root-exported `ShadowConfig` groups `ramp` with the default-off +`logMismatches` diagnostic policy. Runtime overlays merge those two leaves +independently: an omitted leaf inherits its baseline, while +`logMismatches: false` explicitly disables inherited logging. + +The former flat `shadowRamp` field has been removed; migrate it to +`shadow.ramp`. Public construction and static defaults reject the old field, +and a raw runtime provider result containing it fails config resolution and +runs the fallback uncached rather than inheriting defaults. + +## Eligibility + +DialCache schedules shadow work only when all of these conditions hold: + +- the call began inside an enabled DialCache scope; +- a Redis or Valkey adapter is configured; +- normal traversal reaches the remote layer; +- the resolved remote policy has a valid TTL; +- the effective `shadow.ramp` is positive and selects the exact key; +- the configured metrics adapter implements `shadowValidation`; and +- the instance has capacity and no shadow job already owns that exact key. + +A remote serving ramp of `0` is eligible because it preserves a valid remote +policy while excluding the key from serving. + +Missing or invalid remote policy, provider failure, an omitted metrics hook, an +invalid or zero shadow ramp, cohort exclusion, an earlier in-memory hit, or a +disabled call does not start a shadow-only Redis path. + +An invalid runtime `shadow.ramp` does not disturb an otherwise valid +caller-serving Redis hit. DialCache skips shadow work and records a +`config_resolution` error. + +`DialCacheKeyConfig.disabled()` sets the shadow ramp and both serving ramps to +`0`, and sets `shadow.logMismatches` to `false`, so it is the complete kill +switch for new cache invocations. It does not cancel work already admitted. + +An invalid runtime `shadow.logMismatches` does not change the cache result, +shadow result, or terminal shadow metric. For an admitted job, DialCache records +one remote `config_resolution` error and suppresses the warning. + +DialCache validates this diagnostic leaf only after the metrics-hook, +exact-key-cohort, and capacity gates; ineligible, cohort-excluded, and +explicitly dropped work does not report that configuration error. + +### Upgrade note for untracked keys + +Starting in `v0.15.0`, otherwise eligible untracked keys participate in shadow +work. A use case that already had a positive effective `shadow.ramp` can +therefore add source reads, Redis reads and fills, metrics, and opted-in logs. +Set its shadow ramp to `0` before upgrading if that work is not wanted. + +See [Configuration and cache layers](configuration.md) for runtime-overlay +precedence and policy validation. + +## Serving-hit and ramped-down paths + +### Serving Redis hit + +The request path performs its normal Redis read and deserialization in the +key's tracked or untracked mode. It returns that cached value without waiting +for shadow work and retains the exact serialized payload as `C0`. + +On a later unreferenced event-loop turn, the shadow job: + +1. invokes the source loader inside a disabled DialCache context; +2. deserializes `C0` again into an independent cached snapshot; +3. compares that snapshot with the source value `S`; and +4. performs a confirmation read only when the values differ. + +The additional source call must be safe to run for observation. With default +coalescing, one serving Redis leader schedules at most one job for its +followers. With `coalesce: false`, each caller can attempt scheduling; exact-key +shadow deduplication admits at most one concurrent job and reports the others +as `dropped`. + +### Ramped down from Redis serving + +When a valid remote policy excludes the key specifically because its serving +ramp is down, the caller runs and awaits the normal source loader. Shadow work +reuses that same caller-owned promise as `S`; it does not invoke the loader a +second time. + +The detached job reads Redis as `C0` in the key's existing mode. A hit is +compared with `S`. A clean miss can be filled from `S` in that same mode using +the invocation's resolved remote TTL snapshot. + +The shadow Redis value never supplies the caller or populates request-local or +process-local memory. If those in-memory layers are active, only the caller's +source result can populate them. + +When request-local and process-local caching are off and Redis serving is +ramped down, caller invocations remain uncached and do not gain process +coalescing merely because shadowing is enabled. Concurrent same-key callers +can each run the source; shadow deduplication independently admits one shadow +job and reports the others as `dropped`. If an in-memory serving layer is +active, `coalesce: false` likewise keeps each caller's cache path independent. + +## The `C0` / `S` / `C1` algorithm + +`C0` is the original Redis payload: either the payload that served the caller or +the result of the detached ramped-down read. `S` is the successfully accepted +source value. `C1` is an optional confirmation read in the same tracked or +untracked mode as `C0`. + +1. Obtain `C0`. +2. If `C0` is `null`, follow the clean-miss fill path described below. +3. Otherwise, obtain `S`, deserialize a new snapshot from `C0`, and compare the + cached and source values. +4. When they match, emit `match`; no confirmation read is needed. +5. When they differ, read Redis again as `C1` in the same mode, bypassing the + request-local and process-local caches. +6. If `C1` is missing or its bytes differ from `C0`, emit `superseded`. +7. If `C1` is byte-identical to `C0`, emit `mismatch`. + +String payloads compare by their UTF-8 bytes, Buffer payloads compare by bytes, +and a string/Buffer pair with the same UTF-8 bytes is identical for +confirmation. DialCache does not deserialize `C1`, compare it with `S`, or +chase another version. + +`mismatch` therefore means that the exact observed Redis payload survived one +confirmation read after a semantic disagreement. It is not a cross-system +atomic snapshot or a guarantee that the mismatch still exists. For an +untracked key, it is also not proof of primary freshness or invalidation +safety. `superseded` means only that the original observation could not be +confirmed. + +No non-null `C0` is repaired, overwritten, invalidated, or given a refreshed +TTL. That rule also applies when detached deserialization fails. + +### Clean-miss fill + +A clean miss means the semantic Redis read returned `null`. It does not include +a non-null payload that the serializer cannot load. + +On the ramped-down path, DialCache can serialize the caller-accepted `S` and +attempt one ordinary Redis write in the key's existing mode with the resolved +TTL: + +- `filled` means the client returned `true` before the shadow deadline; +- `fill_blocked` means a tracked invalidation watermark returned `false`; and +- `fill_error` means preparing the payload (serialization or compression) or + writing it to Redis failed. + +A source rejection or caller fallback timeout never produces an accepted `S` +and never starts the fill. Once serialization has finished, DialCache checks +the whole-job deadline again before dispatching the write. + +The `C0` read and fill are not atomic. The fill is a normal overwrite, not a +compare-and-set or write-if-still-missing operation. Another writer can +populate Redis after `C0` misses and then be overwritten by the shadow fill. +Tracked writes retain their watermark fence. An untracked fill has no fence and +uses ordinary TTL-based last-writer-wins publication, so an older accepted +source value can overwrite a concurrent newer value and remain until expiry. + +## Comparison semantics + +By default, DialCache uses Node's `util.isDeepStrictEqual`. Plain-object +property insertion order does not affect equality; values, array order, +prototypes, constructors, Buffers, Maps, Sets, and other supported structures +remain strictly compared. + +`shadowComparator(cachedValue, sourceValue)` can define application-level +equality, such as ignoring a volatile refresh timestamp. It is stable +use-case behavior, not runtime rollout policy. The comparator must: + +- return a boolean synchronously; +- be deterministic and side-effect-free; +- avoid mutating either borrowed input; and +- complete in bounded time. + +A throw or non-boolean result is `comparison_error`, not `mismatch`. An +accidental promise is not accepted as a result. DialCache consumes its +settlement so a later rejection is not unhandled, while the job remains +subject to its deadline and capacity rules. + +The cached comparator input is newly deserialized from `C0`; it is not the +object already returned to a serving-hit caller. The source input is the raw +loader result. This deliberately exposes lossy serialization unless a custom +comparator declares that normalization acceptable. + +## Confirmed mismatch logging + +The bounded shadow outcome metric is the default diagnostic. A use case can +separately opt in to one warning for each terminal `mismatch`: + +```ts +new DialCacheKeyConfig({ + shadow: { + ramp: 5, + logMismatches: true, + }, +}); +``` + +Logging does not activate shadow work: the operation must still pass every +eligibility gate, including the required `metrics.shadowValidation` hook. A +warning is emitted only after `C1` is byte-identical to `C0` and the terminal +`mismatch` metric is recorded. Matches, candidates that become `superseded`, +timeouts, dropped work, and errors remain metric-only. + +The warning message is `DialCache shadow validation mismatch`. Its details are: + +| Field | Value | +| --- | --- | +| `cacheNamespace`, `useCase`, `keyType`, `outcome` | Stable metadata; `outcome` is always `"mismatch"`. | +| `cacheKey` | The logical DialCache URN, not the physical Redis storage key; capped at 2 KiB of UTF-8. | +| `cachedValueJson` | Native JSON for the newly deserialized `C0` comparator input; capped at 8 KiB of UTF-8. | +| `sourceValueJson` | Native JSON for the raw source comparator input; capped at 8 KiB of UTF-8. | + +DialCache applies `JSON.stringify` independently to the two values. If it throws +or returns `undefined`, that field is `null` and the other side is still +attempted. + +A byte-clipped string ends in `...[truncated]` within its cap without splitting +a UTF-8 sequence. DialCache never passes the raw compared-value references to +the logger, calls the configured serializer again, or computes a textual diff. + +These bounds limit the fields handed to the logger, not the data DialCache must +inspect to build them: + +- truncation is not redaction; the logical URN can contain ids and arguments, + and either value can contain secrets or personal data; +- native JSON invokes getters and `toJSON`, can omit or normalize unsupported + values, and produces `null` here for cycles, `bigint`, or other failures; +- stringification is synchronous, and the 8 KiB cap applies only after it + returns, so it does not bound traversal, hook execution, event-loop time, or + the intermediate string; and +- metadata, logger framing, and transport escaping are outside the field caps, + so the final event can exceed a sink-specific size limit. + +Enable mismatch logging only for trusted, reasonably bounded values and with an +approved logger, redaction, transport, access, and retention policy. An +unexpected detail-construction failure degrades to the metadata-only warning. +Synchronous logger throws and rejected promises or thenables remain isolated +from cache and shadow correctness. + +## Capacity, deadlines, and detachment + +`shadowMaxInFlight` is a positive safe integer on each `DialCache` instance and +defaults to `1`. It counts scheduled and running jobs. The optional `C1` read +and clean-miss fill remain part of the original slot. + +There is no queue. DialCache emits `dropped` when: + +- another shadow job already owns the exact cache key; or +- the instance has reached `shadowMaxInFlight`. + +Separate instances have separate caps. This is not fleet-wide admission +control for Redis or the source of truth. + +Each job has one monotonic deadline across the detached Redis read, source +result, serializer work, comparison, confirmation read, and clean-miss fill: + +- a finite `fallbackTimeoutMs` is also the whole shadow budget; +- when `fallbackTimeoutMs` is `null`, the caller fallback is unbounded but + shadow work still uses 60 seconds; +- serving-hit timing begins when the detached callback starts; and +- ramped-down timing begins immediately before the caller's source invocation, + including its synchronous prefix. + +Each `C0` or `C1` Redis read also uses the effective +`remoteReadTimeoutMs`. That read deadline bounds DialCache's wait, not the raw +client operation. + +The scheduler, Redis-read timers, and overall shadow timer are unreferenced. +They do not keep an otherwise idle process alive. Detachment is still work on +the Node event loop, not a worker thread; synchronous source, serializer, or +comparator code can occupy the event loop after the request continues. + +Deadline expiry records the applicable outcome, releases retained `C0`, and +prevents later phases from starting. It does not cancel JavaScript promises or +prove that a dispatched Redis command stopped. + +Shadow-owned reads, serializer calls, comparators, and writes can therefore +retain the slot after the DialCache deadline until their underlying work +settles. On the ramped-down path, the shared source promise is caller-owned and +does not retain the shadow slot after the job abandons it. + +Give every source, serializer, Redis client, and telemetry transport a finite +native resource budget. A shadow `timeout` or `fill_error` after write dispatch +does not prove Redis was unchanged. Conversely, `filled` means the client +reported success before the deadline, not that the entry is still present. + +## Data ownership and custom integrations + +Detached execution preserves the original `cached()` argument references or +the `getOrLoad()` loader closure. DialCache cannot generically clone captured +state. Snapshot mutable source-selection inputs before invoking DialCache so +the later source read still describes the already-built cache key. + +Treat the source result `S` as immutable after return. A clean-miss serializer +may inspect it after the caller has continued. + +The effective serializer has additional shadow requirements: + +- `load` can run twice for a sampled serving hit; +- repeated loads of the same payload must be independent; +- `load` must not mutate a borrowed Buffer; and +- asynchronous `load` and `dump` methods need finite application-owned + deadlines. + +A custom `DialCacheRedisClient.read()` must return an operation-owned payload +whose contents remain stable after the method settles. DialCache can retain +that exact `string | Buffer` for detached deserialization and confirmation. +An adapter that pools or recycles response storage must return a dedicated +Buffer. + +A custom metrics adapter must implement the optional `shadowValidation` hook +to admit shadow work. The hook remains optional at the type level so existing +adapters continue to compile; omitting it deliberately keeps shadow execution +off. + +See [Redis and Valkey](redis.md) for the complete custom-client, payload, +deadline, and connection-lifecycle contracts. + +## Consistency modes and race boundaries + +Shadowing preserves the operation's existing Redis mode: + +- **Tracked keys:** `C0` and `C1` use the watermark-aware read protocol, which + atomically checks the value timestamp against the invalidation watermark. + Bundled cluster adapters explicitly route these reads to the primary; a + standalone node-redis client must already target the authoritative endpoint. + A clean-miss fill uses the ordinary tracked write and can be rejected as + `fill_blocked` by a future watermark. +- **Untracked keys:** `C0` and `C1` use the ordinary one-key read route without a + watermark or shadow-specific primary guarantee. A clean-miss fill uses the + ordinary TTL write. Compliant untracked writes do not produce + `fill_blocked`. + +Both modes use the operation's serializer and value TTL. Tracked publication +receives a Redis-time timestamp from the stamp script; an untracked write uses +an informational client-clock timestamp that untracked reads never consult. + +Neither mode makes the initial `C0` read and later fill atomic. For tracked +keys, size `futureBufferMs` to cover the complete source, serialization, client +queue, network, and write interval when stale-publication protection matters. +The watermark fences only the tracked Redis write; it does not synchronously +invalidate request-local or process-local entries. Shadow mode never evicts +those layers. + +See [Targeted invalidation](invalidation.md) for the tracked clock, durability, +retention, and future-buffer contracts. + +### Command amplification + +| Selected path | Added source work | Added Redis work | +| --- | --- | --- | +| Serving Redis hit, semantic match | One observational source read | None beyond the serving read | +| Serving Redis hit, mismatch candidate | One observational source read | One same-mode confirmation read | +| Ramped-down Redis hit, semantic match | None beyond the caller's source read | One same-mode `C0` read | +| Ramped-down Redis hit, mismatch candidate | None beyond the caller's source read | Same-mode `C0` and `C1` reads | +| Ramped-down clean Redis miss | None beyond the caller's source read | One same-mode `C0` read and at most one write | +| Serving Redis miss | None beyond the ordinary path | None beyond the ordinary read and fill | + +Capacity limits bound concurrent jobs, not total work over time. Measure source +and Redis load while increasing `shadow.ramp`. + +## Metrics and compatibility + +Every admitted job, and every job explicitly rejected by deduplication or the +capacity cap, reports one bounded terminal outcome: + +| `outcome` | Meaning | +| --- | --- | +| `match` | The deserialized `C0` and source value matched semantically. | +| `mismatch` | They differed and byte-identical `C1` confirmed the original `C0`. | +| `superseded` | They differed, but `C1` was missing or had different bytes. | +| `filled` | A clean miss was populated successfully before the deadline. | +| `fill_blocked` | A tracked invalidation watermark rejected the clean-miss fill. | +| `fill_error` | Preparing the payload (serialization or compression) or writing the clean-miss fill failed. | +| `redis_error` | The initial detached `C0` read failed or reached its read deadline. | +| `source_error` | The source loader rejected without being the caller's own DialCache fallback timeout. | +| `deserialization_error` | The retained non-null `C0` could not be deserialized. | +| `comparison_error` | The comparator threw or did not return a synchronous boolean. | +| `confirmation_error` | The `C1` read failed or reached its read deadline. | +| `timeout` | The whole shadow deadline expired, including a shared caller fallback timeout. | +| `dropped` | Exact-key deduplication or the per-instance cap rejected the job. | + +Ineligible or cohort-excluded invocations do not emit a shadow outcome. +Outcome labels contain the logical cache namespace, `useCase`, and `keyType`; +they never contain ids, values, payloads, Redis keys, or exception text. The +separately opted-in mismatch warning is value-bearing and does not change this +metric-label contract. + +Redis reads, serializer work, payload sizes, and Redis errors inside detached +jobs use the existing metric hooks with `layer="remote_shadow"`. This keeps +their cost separate from caller-serving `layer="remote"` traffic. The read +that supplied a serving-hit `C0` remains caller-path `remote` work, and a +ramped-down caller still reports `disabled{layer="remote", +reason="ramped_down"}`. + +The dedicated shadow outcome has no `layer` label. A clean `C0` miss also +records the ordinary `miss{layer="remote_shadow"}` before the job's terminal +fill, source, or timeout outcome. + +`MetricLayer` includes `remote_shadow`, and `ShadowValidationOutcome` includes +every value in the table above. TypeScript consumers with exhaustive switches +or `Record` values must handle those cases. Dashboards filtered to +`layer="remote"` intentionally exclude detached cost. + +See [Observability](observability.md) for exact Prometheus and Datadog metric +names, units, labels, and the custom-adapter interface. + +## Rollout checklist + +Before increasing `shadow.ramp`: + +- confirm the source loader is safe to invoke observationally on serving hits; +- use a valid remote TTL with the serving ramp at `0`; +- choose the Redis consistency mode deliberately: for tracked keys, configure a + defensible `futureBufferMs`; for untracked keys, accept TTL-based + last-writer-wins fills without invalidation or a primary-read guarantee; +- verify serializer, comparator, Redis client, source, and telemetry budgets; +- start with a small per-instance capacity and measure `dropped`; +- account for the command amplification above; and +- arrange application-owned shutdown for dependencies that can outlive the + DialCache deadline. + +Keep `shadow.logMismatches` off unless the logged key and values have been +classified and the application has approved their synchronous JSON cost, +redaction, transport, access, and retention policy. + +During rollout, monitor outcome ratios together with detached source latency, +`remote_shadow` Redis errors, fill load, and ordinary source health. Treat +`mismatch` as a signal to investigate value meaning, serialization, key +identity, and source consistency—not as an automatic repair instruction. + +Increase the Redis serving ramp only after the observed cohort, source load, +and invalidation behavior meet the application's acceptance criteria. To stop +new Redis serving and shadow activity through runtime policy, set both the +remote serving ramp and `shadow.ramp` to `0`. + +## Shutdown + +DialCache has no shadow drain or close method. Awaiting promises returned by +`cached()`, `getOrLoad()`, and `invalidateRemote()` drains request-path work, +not detached shadow jobs. + +During shutdown: + +1. stop admitting new DialCache-backed requests; +2. set serving and shadow ramps to `0` if runtime policy remains active; +3. await request-path cache calls and invalidations; +4. use source, Redis-client, serializer, and telemetry-native controls to drain + or terminate their work; +5. close underlying connections after their remaining work drains. + +Unreferenced shadow scheduling means the process may exit before an outcome is +delivered. Already-started source reads, serializers, Redis commands, or +telemetry can still be active, and an already-dispatched fill may have +executed even when its outcome is lost during teardown. Shadow validation is +therefore best-effort during shutdown by design.