From 4a2dc71094a14d7a13eae004ec360e9d18e1f548 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Thu, 23 Jul 2026 10:38:25 -0700 Subject: [PATCH 1/7] docs: focus README on safe cache rollouts --- AGENTS.md | 5 + README.md | 913 ++++++++++++++---------------------------- docs/coalescing.md | 220 ++++++++++ docs/configuration.md | 434 ++++++++++++++++++++ docs/invalidation.md | 205 ++++++++++ docs/maintainers.md | 76 ++++ docs/observability.md | 261 ++++++++++++ docs/redis.md | 377 +++++++++++++++++ 8 files changed, 1868 insertions(+), 623 deletions(-) create mode 100644 docs/coalescing.md create mode 100644 docs/configuration.md create mode 100644 docs/invalidation.md create mode 100644 docs/maintainers.md create mode 100644 docs/observability.md create mode 100644 docs/redis.md diff --git a/AGENTS.md b/AGENTS.md index 64d5883..2aaeca8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,6 +7,8 @@ DialCache is a TypeScript caching library with explicit request-scoped enablemen ## Structure ```text +README.md # Adoption guide, safety model, and reference routing +docs/ # Focused user-facing configuration and operations guides src/ dialcache.ts # Main DialCache API and cached-function wrapper config.ts # Public configuration and rollout types @@ -36,6 +38,9 @@ test/ # Unit and Redis integration tests ## Conventions - Preserve strict TypeScript settings and public abstraction boundaries. +- Keep the README focused on evaluation and adoption. Put complete operational + contracts in a focused `docs/` guide and link it from the relevant README + summary. - Keep Redis client-specific behavior in adapters; core code depends on `DialCacheRedisClient`. - Public exports belong in the root or an explicit integration entry point such as `src/node-redis.ts`, `src/prometheus.ts`, or `src/redis-protocol.ts`. - Use `corepack pnpm` for project commands. diff --git a/README.md b/README.md index 8c4ce1c..e676737 100644 --- a/README.md +++ b/README.md @@ -4,44 +4,80 @@ [![Codecov](https://codecov.io/gh/lan17/DialCache/branch/main/graph/badge.svg)](https://codecov.io/gh/lan17/DialCache) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/lan17/DialCache/badge)](https://scorecard.dev/viewer/?uri=github.com/lan17/DialCache) -Fine-grained TypeScript caching with explicit enabled contexts, request-local memoization, process-local and Redis TTL caching, stable key construction, runtime rollout controls, request coalescing, adapter-based observability, and Redis watermark-based targeted invalidation. +**Roll out backend caching like a feature—not a leap of faith.** + +**DialCache is** a TypeScript library for caching database and service reads +inside Node.js backends. It routes reusable async functions and inline loaders +through one read-through path with request-local memoization, a bounded +in-process LRU, and optional Redis or Valkey caching. + +The “dial” is per-use-case runtime control. Start with caching off, dial the +process-local and remote layers up for stable cohorts of keys, and dial them +back down without changing the loader. + +**DialCache is not** a frontend data cache, cache server, Redis or Valkey +client, or runtime configuration service. It supplies the cache path and +rollout controls; your application still decides what is safe to cache and +owns loader behavior, connections, runtime configuration, keys, TTLs, +invalidation policy, and resource budgets. + +## Safety comes from explicit controls + +- **Off by default.** Outside `dialcache.enable(...)`, both cached wrappers and + inline loaders are true pass-throughs: DialCache does not build a key, + resolve config, access a cache, or coalesce the call. Inside an enabled + scope, a layer still needs an effective policy before it participates. +- **Gradual and reversible rollout.** Configure TTL and ramp independently for + the process-local and remote layers. A ramp of `0` is off, `100` is fully on, + and `DialCacheKeyConfig.disabled()` is the all-layer policy kill switch. +- **Fail-open cache path.** Key, config, cache-read, and serialization-load + failures fall through to the source loader. Cache-write, + serialization-dump, logging, and metrics failures do not replace an otherwise + usable fallback result. Explicit remote invalidation failures are rethrown so + callers never assume a mutation was made safe when it was not. +- **Bounded defaults.** The process-local cache has a 10,000-entry default cap, + active remote reads have a 50-millisecond default deadline, and enabled + fallback executions have a 60-second default deadline. The read deadline + bounds DialCache's wait, not necessarily the underlying Redis command; + applications still need resource-native budgets for client work, config + providers, serializers, and source I/O. + +Use DialCache when you want to: + +- add caching to database or service reads without scattering cache get/set + plumbing across call sites; +- begin with one layer or a small deterministic key cohort, observe it, and + expand or reverse the rollout per use case; +- combine request-local, process-local, and shared caching behind one key and + policy contract; or +- coalesce hot-key misses, invalidate related Redis entries, and emit bounded + cache metrics without rebuilding those mechanisms for every function. ## Contents - [Install](#install) - [Quick start](#quick-start) -- [How caching works](#how-caching-works) -- [Enabled context](#enabled-context) -- [Defining cached functions](#defining-cached-functions) - - [One-shot inline cache blocks](#one-shot-inline-cache-blocks) -- [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions) -- [Runtime config and ramp controls](#runtime-config-and-ramp-controls) -- [Cache layers](#cache-layers) - - [Request-local cache](#request-local-cache) · [Process-local cache](#process-local-cache) · [Redis-backed TTL cache](#redis-backed-ttl-cache) · [Remote read deadlines](#remote-read-deadlines-and-async-liveness) · [Serialization](#serialization) -- [Cached-value ownership](#cached-value-ownership) -- [Targeted invalidation and watermarks](#targeted-invalidation-and-watermarks) -- [Request coalescing](#request-coalescing) - - [Fallback deadlines](#fallback-deadlines) · [Coalescing state](#coalescing-state) -- [Metrics](#metrics) -- [Maintainers](#maintainers) +- [Dial caching up or down](#dial-caching-up-or-down) +- [How the read path works](#how-the-read-path-works) +- [Core concepts](#core-concepts) +- [Production checklist](#production-checklist) +- [Reference guides](#reference-guides) ## Install ```bash pnpm add dialcache -# Choose a Redis client when using the remote layer: -pnpm add redis@~4.7.1 -# or -pnpm add @valkey/valkey-glide -# Add a metrics client only when using its adapter: -pnpm add prom-client@^15.1.3 -# or -pnpm add hot-shots@^17.0.0 ``` DialCache requires Node.js 22.0.0 or newer. Production deployments should use a [currently supported LTS release](https://nodejs.org/en/about/previous-releases). +Redis, Valkey, Prometheus, and Datadog integrations are optional and keep their +clients application-owned: + +- [Redis and Valkey setup](https://github.com/lan17/DialCache/blob/main/docs/redis.md) +- [Prometheus and Datadog setup](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + ## Quick start ```ts @@ -59,678 +95,309 @@ const getUser = dialcache.cached( }, ); -// Caching is OFF outside an enable() scope (see "Enabled context"), so this runs the fn uncached: +// Outside enable(), this is a true pass-through to db.fetchUser: await getUser("123"); -// Inside enable(), reads are cached: +// Inside enable(), the active cache layers participate: const user = await dialcache.enable(() => getUser("123")); ``` -## How caching works - -The wrapped function is the **fallback**: it runs whenever no active cache layer returns a value, whether because layers missed, were disabled, or failed open. - -When caching is enabled, reads flow through: - -```text -request-local cache -> process-local cache -> Redis cache -> fallback function -``` - -- Request-local hits return the value memoized in the current outermost `enable()` scope. -- Results from the lower chain are memoized request-locally when that layer is enabled. -- Process-local hits return immediately. -- Process-local misses try Redis and populate the process-local cache on a Redis hit. -- Redis misses call the fallback and attempt to populate Redis and, when active, the process-local cache. Tracked invalidation may suppress both publications. -- Redis read failures and timeouts are logged, counted in metrics, and fail open without attempting a second Redis operation. Redis write failures also fail open. `invalidateRemote` logs/counts Redis failures and rethrows them so callers do not assume invalidation succeeded. -- Cache-key construction and config-provider failures also fail open and run the fallback uncached. -- A missing effective process-local/Redis TTL disables that layer by policy; a configured TTL with no ramp defaults to 100%. Disabled layers record a disabled reason and fall through to the next layer/fallback. +`cached(fn, options)` preserves the function's parameters and returns a +Promise-based wrapper. `DialCacheKeyConfig.enabled(60)` gives the process-local +and remote layers a 60-second baseline TTL; the remote layer participates only +when a Redis or Valkey client is configured. -Caching as a whole is only active inside an enabled context, described next. +For a one-shot calculation that should remain inline, +[`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) +accepts a +zero-argument loader and a direct key through the same cache contract. -## Enabled context - -Caching is **off by default** and only active inside a `dialcache.enable(...)` scope. This is deliberate: it lets you turn caching **off in write paths** so a stale read can't be cached around a write. DialCache uses Node `AsyncLocalStorage` to keep enabled state scoped to the current asynchronous call chain. - -**Enable once at your request boundary** (e.g. a middleware that wraps read-request handling) so individual call sites don't each need it; wrap mutation handlers in `disable()`: +Enable caching once at a read-request boundary instead of at every call site. +Keep nested mutation work uncached with `disable()`: ```ts await dialcache.enable(async () => { - await getUser("123"); // cached + const user = await getUser("123"); await dialcache.disable(async () => { - await updateUser("123", patch); // reads here are uncached + await updateUser("123", patch); }); - - await getUser("123"); // cached again -}); -``` - -- Default is disabled — `cached()` and `getOrLoad()` calls made **outside** any `enable()` scope simply run their loader uncached (no error), so wrap your read paths to actually cache. -- Enabled state is async-scope-local, not process-global. -- Nested `enable` / `disable` scopes restore the previous behavior when the callback completes. Nested `enable()` calls reuse the outer request-local scope rather than creating a new one. - -## Defining cached functions - -Use `cached(fn, options)` for an extracted, reusable function. The wrapped callable has the same parameters and always returns a `Promise`. For a one-shot calculation that should remain inline, use [`getOrLoad()`](#one-shot-inline-cache-blocks). - -| Option | Required | Description | -| --- | --- | --- | -| `keyType` | yes | The kind of id the key addresses (e.g. `"user_id"`). Together with the id, the invalidation unit for tracked entries. | -| `useCase` | yes | Identifies the individual cache: part of the stored key and the metrics label. | -| `cacheKey` | yes | Selector over `fn`'s parameters; returns a bare id or `{ id, args }`. | -| `defaultConfig` | no | `DialCacheKeyConfig` baseline policy that runtime config overlays field by field (see [Runtime config](#runtime-config-and-ramp-controls)). | -| `serializer` | when the return type is not statically JSON-compatible | Per-function `Serializer` for Redis values (see [Serialization](#serialization)). | -| `trackForInvalidation` | no (default `false`) | Opts this use case's Redis entries into watermark-based targeted invalidation. | -| `fallbackTimeoutMs` | no (default `60_000`) | Fallback deadline in milliseconds, at most 2,147,483,647; `null` disables it (see [Fallback deadlines](#fallback-deadlines)). | - -`cached()` validates `useCase` at registration: a duplicate within one `DialCache` instance throws `UseCaseIsAlreadyRegisteredError`. Both APIs reject the internal name `watermark` with `UseCaseNameIsReservedError`. - -### One-shot inline cache blocks - -`getOrLoad(load, options)` runs one zero-argument loader through the same policy, cache layers, coalescing, invalidation, metrics, serialization, deadlines, and fail-open behavior as `cached()`. It is useful when only part of a larger function should be cached and the loader needs to capture local values: - -```ts -// Reuse the caller-owned defaults; getOrLoad() snapshots them per invocation. -const profileCacheDefaults = DialCacheKeyConfig.enabled(60); - -const profile = await dialcache.getOrLoad( - async () => { - const user = await db.getUser(userId); - return renderProfile(user, locale); - }, - { - keyType: "user_id", - useCase: "BuildProfile", - key: { id: userId, args: { locale } }, - defaultConfig: profileCacheDefaults, - }, -); -``` - -The options match `cached()` except that the direct `key` replaces the `cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and snapshotted for each invocation. Outside an enabled scope, DialCache invokes `load` directly without constructing a key or resolving runtime policy. - -`getOrLoad()` does not register its `useCase`, so repeated calls should reuse one stable, deployment-defined name such as `"BuildProfile"`. Keep it bounded: never derive `useCase` from a user, request, id, or other high-cardinality input because it is part of both cache identity and metrics labels. Put those values in `key` instead. - -Every captured value that can change the result belongs in the bare id or `{ id, args }` key. Concurrent same-key calls may share one caller's in-flight loader and cached value, so all call sites for that identity must also agree on value meaning and serialization. Prefer `cached()` when a loader is reusable; prefer `getOrLoad()` when the calculation is intentionally local to one call site. - -## Keys, ids, and extra dimensions - -For `cached()`, the key comes from the required `cacheKey` selector whose parameters are inferred from `fn`. `getOrLoad()` accepts the same bare id or `{ id, args }` shape directly through `key`: - -The selected or direct key is the value identity contract. It must include every input dimension that can affect the returned value; otherwise distinct calls can reuse the same cached value or share the same in-flight fallback through request coalescing. - -```ts -const searchPosts = dialcache.cached( - (userId: string, page: number, filter: string) => db.searchPosts(userId, page, filter), - { - keyType: "user_id", - useCase: "SearchPosts", - cacheKey: (userId, page, filter) => ({ id: userId, args: { page, filter } }), - defaultConfig: DialCacheKeyConfig.enabled(60), - }, -); -await dialcache.enable(() => searchPosts("u1", 2, "active")); -``` - -`DialCacheConfig.namespace` is the logical cache namespace and the first component of every key. It defaults to `"urn"`, producing keys such as `urn:user_id:123#GetUser`. Set a stable application-specific value when multiple applications may use the same Redis deployment: - -```ts -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, }); ``` -That produces Redis keys beginning with `users-api:...`, or `{users-api:...}` for invalidation-tracked values. `namespace` is DialCache's single cache-identity and key-partitioning setting: it participates in request-local, process-local, Redis, coalescing, deterministic ramp, invalidation, and metrics. It may not contain `{` or `}` because DialCache reserves those characters for Redis Cluster hash tags. Use a namespace to express any required application or environment separation, such as `production-users-api`. - -- **`keyType` + `id` is the invalidation unit for tracked Redis entries.** `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one watermark for that user; any `trackForInvalidation` Redis entry with the same `keyType` and `id` is refreshed across all `args` variants when Redis is read. `invalidateRemote` does not evict existing request-local or process-local entries (see [Targeted invalidation](#targeted-invalidation-and-watermarks)), and untracked Redis entries do not consult the watermark. `useCase` identifies the individual cache (it's the metrics label and part of the stored key). -- **`args` are part of the cache key** — different `args` produce different entries — but invalidation is by `id` only. -- **Scalar key equality is string-based.** Runtime type is not an identity dimension: for matching surrounding dimensions, numeric `1`, string `"1"`, and bigint `1n` identify the same key; argument values `null` and `"null"` also match. `-0` matches `0`, and an `undefined` argument is omitted. If a deployment changes the logical meaning represented by a scalar, change an explicit identity dimension such as `keyType`, `useCase`, or an argument name/value. -- **Non-key inputs** (for example a db handle) are parameters ignored by a `cacheKey` selector or values captured by a `getOrLoad()` loader. They still reach non-coalesced executions, but concurrent same-key cache misses share the leader's execution, so do not omit values like auth context, locale, or cancellation behavior unless sharing one result is correct. -- **Methods:** pass `obj.method.bind(obj)` (or `(...a) => obj.method(...a)`) — a bare `obj.method` reference loses `this`. - -Changing the namespace value intentionally creates a cold-cache boundary across every layer. Old and new keyspaces do not share Redis values or invalidation watermarks. During an overlapping deployment, an invalidation handled by one version is invisible to the other, which can continue serving a stale tracked value until its value TTL expires. If remote invalidation correctness matters, a normal rolling deployment is unsafe: use a coordinated no-overlap cutover, or an operational bridge that prevents both versions from serving remote cache across mutations (for example, temporarily disable and clear remote caching during the transition). After the cutover, provision for fallback/refill load and allow old Redis keys to expire by TTL. - -## Runtime config and ramp controls - -Instance-wide behavior is set through the `DialCache` constructor: - -| `DialCacheConfig` option | Default | Description | -| --- | --- | --- | -| `namespace` | `"urn"` | Logical cache namespace and first key component (see [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions)). | -| `redis` | none | `{ client: DialCacheRedisClient, readTimeoutMs?: number }`; enables the Redis layer with a 50 ms default read deadline (see [Redis-backed TTL cache](#redis-backed-ttl-cache)). | -| `localMaxSize` | `10_000` | Global process-local entry cap; `0` disables process-local storage. Nonnegative safe integer. | -| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the function's `defaultConfig`; `null` applies no overrides. | -| `metrics` | disabled | A `DialCacheMetricsAdapter` (see [Metrics](#metrics)). | -| `logger` | `console` | Receives operational cache failures (`debug`, `warn`, `error`). | - -Per-invocation cache policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` maps keyed by `CacheLayer.LOCAL` (process-local) and `CacheLayer.REMOTE` (Redis), a `requestLocal` boolean, and an optional `remoteReadTimeoutMs`. +Enabled state follows the current asynchronous call chain through Node +`AsyncLocalStorage`; it is not process-global. Nested scopes restore the +previous state when their callbacks settle. -Every cached definition or `getOrLoad()` invocation can provide an optional per-use-case `defaultConfig`. It is the baseline policy, and the `cacheConfigProvider` result is a sparse field-level overlay on that baseline. For cache enablement fields, precedence is runtime config, then `defaultConfig`, then DialCache's disabled baseline. For the remote-read deadline, precedence is runtime `remoteReadTimeoutMs`, `defaultConfig.remoteReadTimeoutMs`, `redis.readTimeoutMs`, then the 50 ms library default. +`disable()` prevents cache access during its callback; it does not evict values +cached before a mutation. Use the appropriate invalidation or TTL policy before +serving later reads of mutable data. -The disabled baseline sets `requestLocal` to false and leaves the process-local and Redis TTLs unset. A shared layer with no effective TTL is disabled by policy. When a shared layer has an effective TTL but no effective ramp, its ramp defaults to 100%. +## Dial caching up or down -`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined` so the overlay can distinguish omission from an explicit `false`; the effective value still defaults to false after resolution. - -A provider result of `null` (or defensive `undefined`) applies no overrides. An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the baseline. Use explicit values to override inherited policy: `requestLocal: false` disables request-local caching and a layer ramp of `0` disables that shared layer. `DialCacheKeyConfig.disabled()` is that explicit kill switch in one call: request-local off and both shared layers ramped to 0. - -DialCache validates `defaultConfig` when `cached()` registers a definition and whenever `getOrLoad()` is invoked: TTLs and remote-read deadlines must be positive safe integers, ramps must be finite percentages from 0 to 100, layer maps must be objects, and `requestLocal` must be a boolean when present. Invalid defaults are rejected immediately. - -Each registration or one-shot invocation captures an immutable internal snapshot of `defaultConfig`; mutating the supplied config or its maps later does not change that operation's baseline. Runtime policy changes belong in the provider's returned overlay. - -Runtime TTL and ramp leaves are used as supplied instead of falling back to valid default leaves. An invalid TTL disables that layer with `invalid_ttl`; a non-finite or nonnumeric ramp disables it with `invalid_ramp`; finite runtime ramps retain the defensive clamp to 0–100. Other layers can still run, and invalid leaves also record a `config_resolution` error so provider garbage is alertable separately from intentional ramp-downs. A malformed runtime config object, layer-map shape, `requestLocal` value, or explicit `remoteReadTimeoutMs` fails config resolution for the invocation, records `config_error`, and executes the fallback uncached without attempting Redis. - -`cacheConfigProvider` is called for every enabled cache invocation before DialCache performs any cache lookup. Keep it cheap, cache any remote/config-store reads inside the provider, and avoid work that would erase the benefit of a cache hit. +Every cache operation can declare a stable `defaultConfig`. An optional +`cacheConfigProvider` returns a sparse runtime overlay for the current key, so +policy can change independently of the loader: ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -const dialcache = new DialCache({ - cacheConfigProvider: async (key) => { - if (key.useCase === "GetUser") { - return new DialCacheKeyConfig({ - // Sparse override: inherit both TTLs and the local ramp from defaultConfig. - ramp: { [CacheLayer.REMOTE]: 25 }, - // Can be changed by the provider at runtime for this use case. - remoteReadTimeoutMs: 35, - }); - } - return null; // apply no overrides; use the cached function's baseline - }, -}); +const runtimePolicies = new Map(); -const getUser = dialcache.cached((userId: string) => db.fetchUser(userId), { - keyType: "user_id", - useCase: "GetUser", - cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ - // Omitted ramps default to 100% because these layers have TTLs. - ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 300 }, - }), +const dialcache = new DialCache({ + cacheConfigProvider: (key) => runtimePolicies.get(key.useCase) ?? null, }); -``` - -`ramp` values are percentages from 0 to 100. `0` disables the layer, `100` enables it, and intermediate values are deterministically sampled by cache key and layer, so the same key is consistently sampled in or out of a partial rollout across calls and instances. The assignment algorithm is owned by DialCache and remains stable across releases. Applications that need an externally coordinated cohort can use `cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. DialCache fetches and resolves one config snapshot per enabled invocation. Provider errors do not activate defaults: they fail open, record `config_error`, and execute the fallback function uncached. - -## Cache layers - -### Request-local cache - -Set `requestLocal: true` to memoize resolved values for the lifetime of the outermost `enable()` scope: -```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; - -const dialcache = new DialCache(); const getUser = dialcache.cached( (userId: string) => db.fetchUser(userId), { keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), + defaultConfig: DialCacheKeyConfig.enabled(60), }, ); -``` - -`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled `CacheLayer`. The `cacheConfigProvider` can turn it on or off for each invocation. `DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and Redis caching, so request-local caching must be selected explicitly. - -DialCache resolves the runtime config once per enabled invocation and uses it for the entire lookup. When the effective `requestLocal` value is false, the invocation skips request-local lookup and storage without deleting an entry already memoized in the scope. A later invocation that enables request-local caching can reuse that entry. - -The outermost `enable()` call owns the request-local lifetime, and nested `enable()` calls reuse that scope. Request-local state is allocated lazily, only when an invocation enables the layer, so scopes that use only process-local or Redis caching do not allocate it. - -Wrap the complete Node HTTP handler so the request-local scope matches the handler's lifetime: - -```ts -import { createServer } from "node:http"; - -const server = createServer((req, res) => { - void dialcache - .enable(async () => { - const user = await getUser(readUserId(req)); - res.setHeader("content-type", "application/json"); - res.end(JSON.stringify(user)); - }) - .catch((error: unknown) => handleRequestError(error, res)); -}); -``` - -Request-local storage has no capacity limit, eviction, or overflow mode. Entries are retained until the outermost `enable()` callback settles. Use it for short-lived scopes with bounded key cardinality; split long-running streams or large batch jobs into smaller scopes when necessary. - -### Process-local cache - -The process-local layer (`CacheLayer.LOCAL`) uses one LRU per `DialCache` instance. It keeps at most 10,000 entries by default across all use cases while retaining each entry's configured TTL. Set `localMaxSize` to a nonnegative safe integer to change the global entry cap; `0` disables process-local storage: - -```ts -const dialcache = new DialCache({ localMaxSize: 25_000 }); -``` - -The limit counts entries rather than estimating JavaScript object memory. Recently read entries stay resident ahead of less recently used entries when the limit is reached. - -### Redis-backed TTL cache - -The Redis layer supports standalone Redis, Valkey, and Redis Cluster. Register DialCache's native node-redis scripts when creating the client, then pass that client to DialCache: - -```ts -import { createClient } from "redis"; -import { DialCache } from "dialcache"; -import { createNodeRedisDialCacheClient, dialcacheRedisScripts } from "dialcache/node-redis"; - -const redisClient = createClient({ - url: process.env.REDIS_URL, - scripts: dialcacheRedisScripts, - disableOfflineQueue: true, - commandsQueueMaxLength: 1_000, - socket: { connectTimeout: 2_000 }, -}); -await redisClient.connect(); - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { - client: createNodeRedisDialCacheClient(redisClient), - // Optional instance default; omit to use DialCache's 50 ms default. - readTimeoutMs: 100, - }, -}); - -async function shutdown(): Promise { - // Stop new work and await every outstanding cached call and invalidation first. - await redisClient.quit(); -} -``` - -`redis.client` is required when Redis is configured and accepts the semantic `DialCacheRedisClient` interface. `redis.readTimeoutMs` is optional and sets the instance default for remote reads; omit it to use 50 ms. Create and connect the underlying client before constructing `DialCache`. Node-redis users should register the supplied scripts and wrap their client with `createNodeRedisDialCacheClient` as shown above. - -Valkey GLIDE users pass an already-created standalone or cluster client and its -module namespace to the GLIDE adapter: - -```ts -import * as valkeyGlide from "@valkey/valkey-glide"; -import { DialCache } from "dialcache"; -import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; - -const glideClient = await valkeyGlide.GlideClient.createClient({ - addresses: [{ host: "127.0.0.1", port: 6379 }], - requestTimeout: 2_000, - advancedConfiguration: { connectionTimeout: 2_000 }, -}); -const redisClient = createValkeyGlideDialCacheClient(glideClient, valkeyGlide); -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: redisClient }, -}); - -function shutdown(): void { - // After draining cached calls and invalidations, release scripts before closing GLIDE. - redisClient.dispose(); - glideClient.close(); -} -``` - -Pass the same module namespace that created the client. DialCache uses its -`Script` constructor and `Decoder.Bytes` value without importing a GLIDE runtime -itself, so linked workspaces and applications with another installed GLIDE -version cannot accidentally mix native script handles. - -The application owns the complete Redis lifecycle. It creates and connects the underlying client and passes the semantic adapter to DialCache. During shutdown, stop starting DialCache-backed work and await every promise returned by a cached function, `getOrLoad()`, or `invalidateRemote()`, including calls still running fallbacks that may later write Redis. A read that crossed DialCache's wait deadline may still be active inside the client, so use client-native telemetry and shutdown controls to drain or terminate that work before disposing adapter-owned resources and closing the connection. DialCache only borrows `redis.client`; it has no close or drain method and never disposes or closes caller resources. - -The node-redis adapter owns no additional resources, so the application closes the underlying node-redis client after draining work. The GLIDE adapter owns five native `Script` handles but not the wrapped connection. After outstanding operations finish, call its idempotent `dispose()` before closing GLIDE as shown above; disposal while an adapter operation is in flight throws rather than releasing a live script. - -Node-redis computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` after `NOSCRIPT`. Its cluster client routes scripts by their first key and performs that fallback on the selected shard. The GLIDE adapter uses GLIDE's native `Script` lifecycle and byte decoder; GLIDE routes scripts from their declared keys. Tracked reads are deliberately routed to primaries so a lagging replica cannot hide an invalidation watermark. - -#### Remote read deadlines and async liveness - -DialCache bounds every active Redis read. The effective timeout is resolved per use case and per invocation: runtime `remoteReadTimeoutMs`, then `defaultConfig.remoteReadTimeoutMs`, then optional instance `redis.readTimeoutMs`, then 50 ms. Values must be positive safe integers no greater than 2,147,483,647. There is no unbounded escape hatch for remote reads. - -When the deadline expires, DialCache aborts the optional `RedisReadContext.signal`, records one `cache_read_timeout` error, logs a `RedisReadTimeoutError`, and starts the source fallback. Late read fulfillment or rejection is consumed and ignored. A read failure or timeout never triggers a post-fallback Redis write; an untracked active process-local miss may retain the source value, while a tracked key suppresses local publication because the failed read did not establish watermark safety. - -Same-key followers share the leader's remaining remote-read budget. The timer covers only the semantic Redis read, not config resolution, serializer load, fallback work, Redis writes, or invalidation. `fallbackTimeoutMs` starts separately when the source fallback begins. - -The bundled node-redis adapter passes the signal through per-command options, which can remove queued work where supported. Aborting after dispatch does not unsend a command or prove that Redis stopped executing it. GLIDE's current script API has no per-invocation signal, so its invocation may continue after DialCache has fallen back. Keep client-native connection, retry, queue, and response budgets in place; they bound underlying resource lifetime while DialCache's deadline bounds caller wait time. - -Writes, invalidations, async `cacheConfigProvider` calls, and custom serializer methods still need finite application-owned budgets. Do not put mutations behind a bare `Promise.race`: rejecting the outer promise neither removes queued work nor proves whether a dispatched mutation executed. - -#### Serialization - -The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not expose client commands or wire encodings. Distinct untracked/tracked read and write Lua sources, the invalidation source, and wire constants are available from `dialcache/redis-protocol`. Custom adapters can throw the root-exported `DialCacheRedisPayloadError`, `DialCacheRedisPayloadEncodingError`, and `DialCacheRedisProtocolError` classes to distinguish malformed payloads, unsupported encodings, and Lua reply-domain violations in logs. DialCache records bounded `cache_read`, `cache_write`, or `invalidation` metrics by failure site. -Redis values use a compact binary frame: - -```text -byte 1 format version -bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) -byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload -``` - -Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is authoritative, so expiry metadata is not duplicated in the frame. `payload` is produced by the operation's serializer, or by `JsonSerializer` by default. Custom serializers can return either `string` or `Buffer`; strings are stored as UTF-8 and Buffers are stored byte-for-byte without base64 expansion. Adapters restore the same representation before calling `serializer.load`. - -DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no runtime validation pass, so the default adds no traversal beyond JSON serialization itself. A top-level `undefined` result is supported with an internal sentinel. - -When `serializer.load` rejects a Redis payload, DialCache records a `serialization_load` error, counts the read as a remote cache miss, runs the fallback, and attempts to replace the rejected payload. A validating custom serializer can therefore treat an incompatible cached value as a refreshable miss without adding a schema version to the cache key. - -`JsonSerializer` validates JSON syntax only. It cannot detect that a structurally valid payload came from an incompatible application value schema. Applications that keep the same `useCase` across deployments must keep default-JSON values backward compatible. For an incompatible change, either provide a serializer whose `load` method validates and rejects the old shape, or change `useCase` to isolate the new cache entries. During a mixed deployment, mutually incompatible validating serializers can repeatedly reject and replace each other's values; correctness is preserved, but expect additional fallback and Redis-write load until the rollout converges. - -When a cached function or inline loader's resolved return type is statically JSON-compatible, `serializer` remains optional. This includes JSON primitives, arrays, plain object/interface shapes, optional object fields, and a top-level `undefined`. Types known not to survive the default round trip require a typed `Serializer`: - -```ts -import { DialCache, type Serializer } from "dialcache"; - -const dialcache = new DialCache(); -const dateSerializer: Serializer = { - dump: (value) => value.toISOString(), - load: (value) => new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), -}; - -const getUpdatedAt = dialcache.cached( - (userId: string) => db.fetchUpdatedAt(userId), - { - keyType: "user_id", - useCase: "GetUpdatedAt", - cacheKey: (userId) => userId, - serializer: dateSerializer, - }, +// Start with the local 10% ramp cohort; keep the remote layer off. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 10, + [CacheLayer.REMOTE]: 0, + }, + }), ); -``` - -The compile-time guard rejects known incompatible shapes such as `Date`, `Map`, `Set`, `bigint`, symbols, functions, Buffers, typed arrays, method-bearing class instances, required nested `undefined`, `unknown`, and `any`. It applies to every `cached()` declaration and `getOrLoad()` invocation because active layers are selected at runtime. A global Redis serializer is not parameterized by each returned type, so it cannot discharge this requirement; non-JSON operations must select a typed serializer. - -This guard is deliberately conservative and is not a proof of runtime data. TypeScript cannot detect non-finite numbers, cyclic/shared references, runtime getter or `toJSON` behavior, or data-only class instances that look like plain objects. Opaque, generic, or deeply recursive types may also require an explicit serializer. Providing `Serializer` (including an explicitly typed `JsonSerializer`) is a trusted caller assertion; DialCache does not serialize-and-deserialize again to validate it. - -## Cached-value ownership - -Treat values returned by cached functions or `getOrLoad()` as immutable. DialCache does not clone or freeze values stored in request-local or process-local memory. Mutating a cached object can therefore be observed by later callers in the same request, callers in other requests that hit the process-local cache, or callers that coalesced onto the same in-flight result. - -This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed arrays, and class instances. Redis deserialization can produce a different reference from an in-memory hit, so reference identity is layer-dependent and is not part of the API contract; never rely on a specific layer cloning a value before mutation. - -If a caller needs a mutable value, copy it explicitly before changing it: - -```ts -const sharedUser = await getUser("123"); -const editableUser = structuredClone(sharedUser); -editableUser.displayName = "New name"; -``` - -Use a narrower copy when its semantics are sufficient; the ownership boundary is the caller's responsibility. - -## Targeted invalidation and watermarks - -Mutable Redis-backed use cases can opt into targeted invalidation by setting `trackForInvalidation: true` in the options and calling `dialcache.invalidateRemote(keyType, id, futureBufferMs)` after writes. The buffer is an application-owned safety value; DialCache cannot choose a universally safe nonzero value: -```ts -import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; -import { createNodeRedisDialCacheClient } from "dialcache/node-redis"; - -const dialcache = new DialCache({ - namespace: "users-api", - redis: { client: createNodeRedisDialCacheClient(redisClient) }, -}); - -// Chosen from this application's clock-skew bound and measured worst-case source/fallback timings. -const USER_INVALIDATION_BUFFER_MS = 5_000; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetMutableUser", - cacheKey: (userId) => userId, - trackForInvalidation: true, - // Strongly invalidated mutable data should disable request-local and process-local caching. - defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.REMOTE]: 300 }, - ramp: { [CacheLayer.REMOTE]: 100 }, - }), - }, +// Later, ramp both shared layers to 100%. +runtimePolicies.set( + "GetUser", + new DialCacheKeyConfig({ + ramp: { + [CacheLayer.LOCAL]: 100, + [CacheLayer.REMOTE]: 100, + }, + }), ); -await updateUser("123", patch); -await dialcache.invalidateRemote("user_id", "123", USER_INVALIDATION_BUFFER_MS); +// Reverse the rollout without changing getUser. +runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); ``` -Invalidation writes a Redis watermark at `{encodedNamespace:encodedKeyType:encodedId}#watermark`. Tracked Redis cache entries use the same Redis Cluster hash tag, for example `{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1`, so the value key and watermark key live in the same slot. Key components are percent-encoded before joining so delimiters inside IDs or args cannot collide with delimiters in the key format. Components may not contain `{` or `}` because those characters would corrupt the hash tag. - -The internal `:dialcache-frame-v1` suffix identifies values written with DialCache's binary protocol. Watermarks are stored as decimal timestamps. +In production, the provider can read from an application-owned dynamic config +client instead of an in-memory map. DialCache resolves one policy snapshot per +enabled invocation. Keep the provider cheap and give any asynchronous work its +own finite budget. -A cached Redis value whose Redis-created timestamp is older than or equal to the watermark is treated as stale and refreshed through fallback. `invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the greater of its existing value and Redis's current time plus the buffer. While that future window is active, an invocation that reaches the tracked Redis read treats the covered value as a miss. If its fallback then reaches the tracked Redis write, Redis rejects the write and DialCache also suppresses the corresponding process-local population; the fallback value still returns to its caller. Request-local memoization remains unconditional, and invocations whose remote layer is disabled or ramped out do not consult the watermark and are not fenced by it. +For the process-local and remote layers: -The bundled timestamp protocol assumes that system clocks are synchronized across every Redis node eligible for primary promotion. Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache does not detect or compensate for cross-node clock skew. If this deployment assumption is violated, failover can temporarily suppress tracked cache fills or allow a pre-invalidation value to remain readable until it expires or a later invalidation advances the watermark past its timestamp. +- a missing effective TTL disables that layer by policy; +- a configured TTL with no ramp defaults to `100`; +- `0` disables the layer; +- `100` enables the layer for every key; and +- an intermediate ramp uses DialCache's deterministic key-and-layer + assignment. -Watermarks are invalidation state, not disposable cache entries. The Redis deployment must preserve them for their derived TTL: use `noeviction` or an equivalent guarantee for deployments that rely on the publication fence, and choose persistence and failover guarantees appropriate to the application's consistency requirements. A missing watermark makes tracked reads miss, but a later tracked write cannot distinguish an empty cache from lost invalidation history; it creates a new baseline watermark and can publish fallback data that the lost future watermark would have rejected. Redis replication is asynchronous by default, and DialCache does not issue `WAIT` or provide strong consistency across failover. +Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% +of calls. Increasing or decreasing a ramp preserves membership for keys that +remain inside the threshold, and local and remote cohorts are layer-specific. +DialCache keeps the assignment stable across releases. -Tracked writes create a baseline watermark and extend its TTL to at least the value TTL plus one minute. Neither tracked writes nor invalidation shorten a longer or persistent watermark TTL; invalidation extends it to at least the remaining future-buffer window plus one minute. There is no fixed watermark retention floor, and reads do not extend watermark lifetime. +If an application needs an externally coordinated cohort, its +`cacheConfigProvider` can return a per-key ramp override of `0` or `100`. +Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing +entries rather than deleting them; a later ramp-up can reuse entries that +remain valid. -`futureBufferMs` must be a nonnegative safe integer. The default is zero, but zero provides no stale-publication protection once Redis time advances. Every production invalidation should pass a named, application-owned nonzero value based on that application's measured or conservatively bounded timings; there is no universally safe library value. +Request-local caching is controlled separately by the `requestLocal` boolean. +`DialCacheKeyConfig.disabled()` sets it to `false` and ramps both shared layers +to `0`. Provider errors do not silently activate the baseline: the invocation +records a config error and runs the source loader uncached. -Size the buffer to cover the maximum expected negative clock skew between promotion-eligible Redis nodes plus the complete interval in which stale data could still reach the Redis write: source visibility or replication lag, the full remaining tail of any fallback that may already have observed the pre-mutation value, `serializer.dump`, Redis client queue and network latency, Lua script execution, the write itself, and a safety margin. Invalidate only after the source mutation commits. Underestimating this interval can allow a delayed stale fallback to repopulate Redis after the watermark window ends. Overestimating it lengthens the tracked Redis miss/write-suppression window described above, increasing fallback load without publishing stale values. A larger buffer does not delay or suppress returning fallback values to callers. +Remote-read waiting is runtime-controlled too. An overlay +`remoteReadTimeoutMs` takes precedence over the operation's `defaultConfig`, +then the instance's `redis.readTimeoutMs`, then the 50-millisecond core default. +Remote reads always have a finite positive deadline. -This is a timing contract rather than a cancellation or acquisition fence: the buffer prevents stale fallback results from passing that tracked Redis write only while the configured window remains active, and it does not force a fallback to read from an authoritative source. +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for sparse-overlay precedence, validation, and layer behavior. -Targeted invalidation is remote-only and enforced by Redis watermarks. `invalidateRemote` does not evict existing request-local or process-local entries. Strongly invalidated mutable data should disable request-local and process-local caching (or use a very short process-local TTL only when stale reads are acceptable). +## How the read path works -## Request coalescing +Inside an enabled scope, active layers are checked in order: -DialCache coalesces in-flight work at the lifetime of the first active cache layer: - -- When request-local caching is enabled, same-key callers in one outermost `enable()` scope share request-scoped in-flight work before the request-local lookup. Its resolved value is then memoized for later sequential calls in that scope. -- When process-local or Redis caching is enabled, same-key callers share in-flight work within one `DialCache` instance before the first active shared layer. This is reported as `scope="process"`, still applies when request-local caching is off, and can combine leaders from separate request scopes using the same instance. - -```ts -await dialcache.enable(async () => { - // Same cold key, concurrent calls: one fallback execution, one shared result. - const [a, b] = await Promise.all([getUser("456"), getUser("456")]); -}); -``` - -With Redis configured, an instance-scoped leader that misses the process-local cache runs one bounded Redis read and, on a normal miss, the fallback/cache write; followers share its remaining read budget and await the same result. Process-local-only misses share the leader's fallback/cache write. This protects Redis and the source of truth from a thundering herd on hot keys. - -Coalescing only applies when at least one cache layer is active. Calls outside `enable()` are true pass-through. Calls where request-local, process-local, and Redis are all disabled are uncached and uncoalesced, but because they were initially enabled, the fallback deadline below still applies. - -Because coalescing is keyed by the selected or direct key, concurrent calls with the same key share the leader's execution. Any function argument or captured value omitted from the key must be safe to share this way; include inputs such as locale, auth context, or cancellation behavior when they can change the returned value or whether the underlying loader should run separately. - -### Fallback deadlines - -Once an initially enabled invocation starts its fallback, DialCache applies a 60-second monotonic deadline by default. Set `fallbackTimeoutMs` once on a cached wrapper or on each `getOrLoad()` invocation to choose a positive integer deadline in milliseconds, up to 2,147,483,647, or set it to `null` to preserve an intentionally unbounded fallback: - -```ts -import { FallbackTimeoutError } from "dialcache"; - -const getUser = dialcache.cached( - (userId: string) => db.fetchUser(userId), - { - keyType: "user_id", - useCase: "GetUserWithDeadline", - cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), - fallbackTimeoutMs: 2_000, - }, -); - -try { - await dialcache.enable(() => getUser("123")); -} catch (error) { - if (error instanceof FallbackTimeoutError) { - logger.warn("source lookup exceeded its DialCache budget", { - useCase: error.useCase, - timeoutMs: error.timeoutMs, - }); - } -} -``` - -The timer starts only when the fallback begins, including after a remote-read deadline has elapsed. Same-key followers share the process or request-local leader's remaining budget and receive its `FallbackTimeoutError`; pass-through invocations where every layer is disabled have independent timers. Cache hits create no fallback timer. Calls that were initially outside an enabled context remain true pass-through and are not timed out, even when the operation configures `fallbackTimeoutMs`. - -Deadline delivery requires the JavaScript event loop to make progress. It cannot preempt a synchronous fallback prefix or other event-loop blocking, so rejection can arrive later than the configured duration; when control returns, DialCache checks the monotonic deadline before accepting the result. The deadline timer remains referenced until the fallback settles or times out. Consequently, an abandoned enabled fallback can keep an otherwise idle short-lived process alive until that deadline; shutdown code should drain outstanding DialCache work rather than discarding its promises. - -Timing out rejects the DialCache chain and clears its flight normally. A later fallback resolution is ignored, so that timed-out invocation cannot proceed to serializer, Redis, or local-cache publication. The underlying function is not canceled and may continue its own I/O or side effects; give the source operation its own native timeout or `AbortSignal` whenever possible. `fallbackTimeoutMs: null` disables this guard and makes finite fallback settlement entirely application-owned. Use the `null` escape hatch only after intentionally accepting that liveness risk. - -Timeout failures retain the bounded metrics classification `error="fallback"` with `in_fallback="true"`; the typed error provides the timeout details without adding high-cardinality labels. - -### Coalescing state - -`getCoalescingState()` returns a detached, point-in-time snapshot of process-scoped flights owned by that `DialCache` instance: - -```ts -const state = dialcache.getCoalescingState(); - -state.process.activeLeaders; -state.process.activeFollowers; -state.process.oldestLeaderAgeMs; // null when idle +```text +request-local -> process-local LRU -> Redis or Valkey -> source loader ``` -A leader is one exact cache key currently tracked by the instance-scoped coalescer. A follower is each later invocation that joined that pending leader; the initiating invocation is not counted as a follower. Followers remain counted until their leader settles because abandoning a JavaScript promise is not observable. Request-local flights are deliberately excluded because their lifecycle is bounded by the outer `enable()` scope. `oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is requested. +The wrapped function or inline loader is the fallback and remains the source of +the returned value when every active layer misses or a cache operation fails +open. -There is no library-wide flight cap or age-based replacement. A registry cap would bound only DialCache metadata while overflow or eviction could still create unbounded source work and unsafe duplicate publication. Finite operation deadlines provide eventual cleanup; application admission control and backpressure remain responsible for bounding simultaneous distinct-key work. Monitor leader count and oldest age to verify that those budgets hold in production. +- A request-local hit returns the value memoized in the current outermost + `enable()` scope. +- A process-local hit returns from the `DialCache` instance's bounded LRU. +- A process-local miss can read Redis and populate the process-local cache. +- A remote miss runs the fallback and attempts to populate active shared + layers. +- A remote read failure or timeout runs the fallback without a second Redis + operation. An untracked result may still populate process-local cache; a + tracked result does not, because watermark safety was not established. +- Same-key concurrent work is coalesced at the lifetime of the first active + layer. -## Metrics +When all layers are disabled by policy, an initially enabled call remains +uncached and uncoalesced, but its fallback deadline still applies. A call that +started outside an enabled scope remains a true pass-through and does not get a +DialCache deadline. -Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the constructor. `new DialCache()` does not import a metrics backend, register collectors, or emit metrics. +## Core concepts -### Prometheus +### Cache operations and keys -Install `prom-client` separately, create the registry your application owns, and pass the explicit Prometheus adapter to DialCache: +`cached(fn, options)` defines both a callable and the value-identity contract: -```bash -pnpm add prom-client@^15.1.3 -``` +| Option | Required | Purpose | +| --- | --- | --- | +| `keyType` | yes | Names the kind of id and, with `id`, the invalidation unit for tracked Redis entries. | +| `useCase` | yes | Identifies this individual cache in stored keys and metrics. | +| `cacheKey` | yes | Selects the bare id or `{ id, args }` from the function parameters. | +| `defaultConfig` | no | Supplies the baseline policy overlaid by runtime config. | +| `serializer` | for statically non-JSON return types | Defines the Redis representation for this operation's value. | +| `trackForInvalidation` | no | Opts the remote entries into watermark-based targeted invalidation. | +| `fallbackTimeoutMs` | no | Sets the fallback deadline; defaults to `60_000`, and `null` disables it. | + +Use `getOrLoad(load, options)` when a one-shot calculation should remain inline. +It follows the same cache, policy, coalescing, invalidation, serialization, and +deadline contracts, but takes a direct `key` instead of a `cacheKey` selector. +It does not register `useCase`, so repeated calls should reuse one stable, +deployment-defined name. + +The selected or direct key must include every input dimension that can affect +the returned value. Same-key concurrent calls may share the leader's execution, +so ignored function arguments or captured values such as auth context, locale, +or cancellation behavior must truly be safe to share. + +Set a stable, application-specific `namespace` when applications or +environments share Redis: ```ts -import { Registry } from "prom-client"; -import { DialCache } from "dialcache"; -import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; - -const registry = new Registry(); const dialcache = new DialCache({ - namespace: "users-api", - metrics: createPrometheusDialCacheMetrics({ - registry, - prefix: "myapp_", // myapp_dialcache_request_counter, etc. - }), -}); - -app.get("/metrics", async (_req, res) => { - res.type(registry.contentType).send(await registry.metrics()); + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, }); ``` -The adapter requires a caller-owned `Registry`; it never uses the global default registry and does not clear or otherwise own the registry lifecycle. Multiple adapters with the same registry and prefix reuse existing collectors when their type, help, labels, histogram buckets, and exemplar mode match. Adapter construction fails before registering anything if a same-name collector has an incompatible schema; use a unique prefix or a separate registry to resolve the collision. +See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) +for key encoding, secondary arguments, namespace changes, serializers, and +value ownership. -The Prometheus adapter emits: +### Cache layers -| Metric | Type | Labels | Description | -| --- | --- | --- | --- | -| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips (`context`, `policy_disabled`, `invalid_ttl`, `invalid_ramp`, `ramped_down`, `config_error`) | -| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors classified by a bounded failure site | -| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by `request_local` or `process` scope | -| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency | -| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | +| Layer | Scope | Primary use | +| --- | --- | --- | +| Request-local | Outermost `enable()` scope | Memoize repeated reads during one bounded request or job. | +| Process-local | One `DialCache` instance | Serve hot values from a bounded in-process LRU. | +| Redis or Valkey | Shared remote store | Reuse TTL-cached values across processes and hosts. | -`policy_disabled` means that a process-local or Redis layer has no effective TTL after runtime overlays are applied. It is an intentional policy outcome, including the default when `defaultConfig` is omitted, rather than a configuration-loading failure. +Each invocation uses one resolved policy snapshot for all three layers. +Request-local storage has no capacity limit, so use it only for short-lived +scopes with bounded key cardinality. Process-local values count toward one +instance-wide entry cap. Remote values use a serializer selected by the cache +operation or the Redis configuration. -Every metric carries `cache_namespace`, including disabled-context, key-construction, coalescing, and invalidation paths that do not have a constructed key. Its value is `DialCacheConfig.namespace`, defaulting to `urn`. The `layer` label is `request_local`, `local` (process-local), or `remote`. Disabled-context, key-construction, and config-provider failures use `noop` because no cache layer was reached. The bounded `scope` label on `dialcache_coalesced_counter` distinguishes request-local from instance-scoped single-flight work. `scope="process"` coordinates calls only within one `DialCache` instance; separate instances in the same process do not share in-flight state. +Cached in-memory values are shared by reference. Treat every returned value as +immutable, or copy it explicitly before mutation. -### Datadog +### Targeted invalidation -Install `hot-shots` separately, create the DogStatsD client your application owns, and pass it to the Datadog adapter: - -```bash -pnpm add hot-shots@^17.0.0 -``` +Mutable Redis-backed use cases can opt into watermark-based invalidation with +`trackForInvalidation: true`, then call: ```ts -import StatsD from "hot-shots"; -import { DialCache } from "dialcache"; -import { createDatadogDialCacheMetrics } from "dialcache/datadog"; - -const dogStatsD = new StatsD({ - host: process.env.DD_AGENT_HOST, - globalTags: { service: "users-api", env: process.env.DD_ENV ?? "development" }, - errorHandler: (error) => logger.warn("DogStatsD error", { error }), -}); - -const dialcache = new DialCache({ - namespace: "users-api", // cache identity and cache_namespace tag - metrics: createDatadogDialCacheMetrics({ - client: dogStatsD, - observationMetricType: "distribution", - namespace: "dialcache", // metric-name prefix: dialcache.request.count, etc. - }), -}); - -// After outstanding cache operations finish during application shutdown: -dogStatsD.close(); -``` - -`hot-shots` is the supported and tested client, but the adapter depends only on the exported `DatadogDogStatsDClient` structural interface. DialCache does not import or install `hot-shots`, create a client, flush buffers, close sockets, or otherwise own the client lifecycle. - -`observationMetricType` is required. `"distribution"` is recommended when latency and size percentiles must aggregate across hosts; enable the desired distribution percentiles and aggregations in Datadog. Choose `"histogram"` when host-level histogram aggregation matches your existing Datadog setup. The choice applies uniformly to all four duration/size metrics. Both modes produce Datadog custom metrics. Distribution volume scales with unique tag-value combinations: Datadog counts five baseline aggregations per combination, and enabling percentile aggregations adds five more. Review [Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) before rollout. Do not send both types under the same namespace: when changing types, use a new namespace during migration so one metric identity never mixes histogram and distribution points. - -`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to `dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache namespace emitted as the `cache_namespace` tag. The Datadog metric namespace must start with a letter and contain only letters, numbers, underscores, and dot-separated non-empty segments. The adapter rejects invalid metric namespaces and final metric names longer than 200 characters rather than relying on client-side normalization. A `hot-shots` `prefix` is applied after the adapter constructs the name, so include that prefix when checking the final length and avoid combining it with the metric namespace accidentally. Client-level `globalTags` are appended by `hot-shots`; the table below lists the tags added by the adapter. - -The Datadog adapter emits exact increments of `1` for counters and preserves seconds and bytes without unit conversion: - -| Metric | Type | Tags | Description | -| --- | --- | --- | --- | -| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | -| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | -| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors by bounded failure site | -| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | -| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | -| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | -| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the underlying function settles or timeout rejection is delivered | -| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | -| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | - -Synchronous client throws are isolated by DialCache's fail-open metrics boundary. Buffered transport failures happen outside that synchronous call, so configure the DogStatsD client's error handling and shutdown behavior as part of application ownership. - -### Error categories - -The `error` label reports where an operation failed rather than copying the thrown value's class or `Error.name`: - -| `error` | Meaning | -| --- | --- | -| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | -| `config_resolution` | Runtime or layer configuration, or ramp resolution, failed | -| `cache_read` | A local-cache or Redis read failed | -| `cache_read_timeout` | A Redis read exceeded its effective remote-read deadline | -| `cache_write` | A local-cache or Redis write failed | -| `serialization_load` | Deserializing a Redis payload failed | -| `serialization_dump` | Serializing a value for Redis failed | -| `invalidation` | Writing an invalidation watermark failed | -| `fallback` | The wrapped application function failed or exceeded its DialCache deadline | -| `unknown` | Reserved for an otherwise unclassified future failure site | - -These values are defined by the backend-neutral core and are identical for every metrics adapter. Raw thrown values, error names, messages, timeout values, cache IDs, arguments, and Redis keys are never included in metric labels. Operational errors are still passed to the configured logger where the existing failure path logs them. `in_fallback` remains the explicit cache-plumbing-versus-application distinction. - -### Custom adapters - -For other telemetry backends, implement `DialCacheMetricsAdapter` and pass the adapter through `new DialCache({ metrics })`. Every backend-neutral label object exposes the logical namespace as camel-case `cacheNamespace`; adapters should map it to their backend's `cache_namespace` label/tag. This field is present even when no key or cache layer was reached. Synchronous adapter failures are isolated from cache behavior and application fallbacks. Omit `metrics` to disable metrics. - -## Maintainers - -### Cache-path benchmark - -From a repository checkout, run the semantic microbenchmark after installing dependencies: - -```bash -pnpm benchmark:request-local +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); ``` -The command builds `dist` before reporting six scenarios: sequential request-local hits, sequential process-local hits, enabled bounded fallbacks, request-local coalescing fan-out, process coalescing fan-out, and remote-read-deadline coalescing fan-out. The benchmark is a maintainer tool and is not included in the published package. It asserts fallback counts, coalescing state, timer cleanup, and returned values but deliberately applies no timing threshold. Override its work sizes with `DIALCACHE_BENCH_ITERATIONS` and `DIALCACHE_BENCH_FANOUT`. - -### Releasing - -Publishing starts by manually running the `Release` workflow from current `main`. After the package checks pass, Semantic Release selects the next version from Conventional Commits since the highest stable `vX.Y.Z` tag. Breaking changes bump major, `feat` bumps minor, and every other normal PR-title type (`fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, `chore`, `ci`, and `revert`) bumps patch. The highest required bump wins. - -The workflow opens a `release: ` PR whose only change is the matching `package.json` version. `release` is a reserved Conventional Commit type configured not to request another release, so the version-control commit does not cause an extra bump. GitHub marks workflow runs for a PR opened with `GITHUB_TOKEN` as approval-required; approve those runs, review the PR, and squash-merge it normally through the protected branch. - -The merge triggers the publish job. Before any release side effect, it verifies current `main`, the release commit subject, the one-file diff, the package version, the absent tag, and Semantic Release's independently calculated version and commit. It then reruns the package checks and asks Semantic Release to create the matching Git tag, publish the public npm package with provenance, and publish the GitHub release. - -The repository must enable **Allow GitHub Actions to create and approve pull requests** under Actions workflow permissions. This workflow uses that capability only to create the version PR; it never approves or merges one, and no ruleset bypass actor or persistent release credential is required. +Invalidation is deliberately remote-only. It does not evict existing +request-local or process-local values, so strongly invalidated mutable data +should disable those layers or tolerate their TTL-bounded staleness. + +The buffer must be a named, application-owned nonzero value sized for clock +skew and the full stale-work window. See +[Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) +before enabling it in production. + +### Request coalescing and fallback deadlines + +Concurrent callers with the same cache key share active work within the first +active cache scope: one outer request for request-local caching, or one +`DialCache` instance for the shared layers. This mitigates hot-key stampedes +inside that scope; it is not cross-process coordination. + +Same-key followers share the leader's remaining remote-read budget. The +fallback deadline starts separately only if and when the source loader begins. + +Enabled fallbacks have a 60-second monotonic deadline by default. Timing out +rejects the DialCache chain and prevents the late result from being published, +but it does not cancel the underlying function. Give source operations their +own native timeout or `AbortSignal`. + +See [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) +for exact sharing, deadline, cleanup, and admission-control contracts. + +### Observability + +Metrics are disabled unless a `DialCacheMetricsAdapter` is supplied. First-party +adapters support caller-owned Prometheus registries and Datadog DogStatsD +clients. Bounded labels report layer requests, misses, disabled reasons, +coalescing scopes, serialization work, and cache versus fallback failures. + +See [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) +for installation, collector schemas, metric names, and custom adapters. + +## Production checklist + +Before ramping a use case: + +- enable DialCache only around read paths, and keep mutation paths inside + `disable()` or outside the enabled boundary; +- verify that every selected or direct key includes each value and execution + dimension that is unsafe to share; +- begin at `0` or a small deterministic key cohort, monitor source load, cache + errors, hit rate, latency, remote-read and fallback timeouts, and coalescing + state, then increase in controlled steps; +- keep a runtime path to `DialCacheKeyConfig.disabled()`; +- choose an effective DialCache remote-read deadline, and configure + resource-native budgets for the underlying Redis work, config providers, + serializers, and source operation; +- use a conservative `localMaxSize` and bounded request-local scopes; +- treat cached values as immutable; +- verify serializer compatibility across mixed application versions; and +- for tracked invalidation, synchronize promotion-eligible Redis clocks, + preserve watermark keys for their derived TTL with `noeviction` or an + equivalent guarantee, choose suitable persistence and failover behavior, and + size a nonzero buffer from measured or conservatively bounded timings. + +## Reference guides + +- [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) — definitions, keys, + runtime overlays, request-local and process-local behavior, and value + ownership. +- [Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) — node-redis and GLIDE setup, lifecycle, + liveness, binary protocol, and serialization. +- [Targeted invalidation](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md) — watermarks, Redis Cluster + placement, clock assumptions, and buffer sizing. +- [Coalescing and fallback liveness](https://github.com/lan17/DialCache/blob/main/docs/coalescing.md) — sharing scopes, + deadlines, state inspection, cleanup, and backpressure. +- [Observability](https://github.com/lan17/DialCache/blob/main/docs/observability.md) — Prometheus, Datadog, metric schemas, + error categories, and custom adapters. +- [Maintainer guide](https://github.com/lan17/DialCache/blob/main/docs/maintainers.md) — benchmarks and the protected release + workflow. + +DialCache is licensed under the +[MIT License](https://github.com/lan17/DialCache/blob/main/LICENSE). diff --git a/docs/coalescing.md b/docs/coalescing.md new file mode 100644 index 0000000..4ce4c34 --- /dev/null +++ b/docs/coalescing.md @@ -0,0 +1,220 @@ +# Coalescing and fallback liveness + +[Back to the README](../README.md) + +DialCache shares same-key in-flight work within the lifetime of the first active +cache layer. It applies a finite deadline to each active remote read and a +separate default deadline once an initially enabled invocation begins its +fallback loader. + +These mechanisms reduce duplicate source work and give active flights eventual +cleanup. They do not replace cross-process coordination, source-native +cancellation, application admission control, or backpressure. + +## Request coalescing + +DialCache has two sharing scopes. + +### Request-local scope + +When request-local caching is active, callers with the same key in one outermost +`enable()` scope share in-flight work before the request-local lookup. + +The resolved value is memoized for later sequential calls in that scope. A +different outer request has a different request-local flight registry. + +### Process scope + +When process-local or remote caching is active, same-key callers share work +within one `DialCache` instance before the first active shared layer. + +This is reported as `scope="process"`, but it is instance-scoped: + +- separate requests using the same `DialCache` instance can share; +- separate `DialCache` instances in one process do not share; and +- separate processes or hosts do not share. + +```ts +await dialcache.enable(async () => { + // Same cold key and active shared layer: + // one fallback execution, one shared result. + const [first, second] = await Promise.all([ + getUser("456"), + getUser("456"), + ]); +}); +``` + +With a remote layer configured, an instance-scoped leader that misses +process-local cache performs one bounded Redis read. Followers share that read +and its remaining deadline. On a remote miss, the leader runs the fallback and +cache write; followers await that result. + +For a process-local-only miss, followers share the leader's fallback and local +write. This mitigates a thundering herd on one hot key within the instance. + +## When calls do not coalesce + +Coalescing applies only when at least one cache layer is active: + +- calls that start outside `enable()` are true pass-through; +- initially enabled calls with every layer disabled are uncached and + uncoalesced; and +- process-scoped work is never shared across `DialCache` instances. + +An initially enabled all-disabled call still receives the fallback deadline +described below. + +Because coalescing is keyed by the full constructed cache key, concurrent calls +with the same identity share the leader's execution. Every function argument +or captured value omitted from the selected or direct key must be safe to share +this way. + +Include locale, auth context, cancellation behavior, or any other input in the +key when it can change: + +- the returned value; +- whether the underlying function should run independently; or +- whether two callers may safely share one result. + +## Fallback deadlines + +Once an initially enabled invocation begins its wrapped fallback, DialCache +applies a 60-second monotonic deadline by default. + +Set `fallbackTimeoutMs` on a cached wrapper or `getOrLoad()` invocation to +choose a positive integer deadline in milliseconds, up to 2,147,483,647. Set +it to `null` only when the application intentionally accepts an unbounded +fallback: + +```ts +import { FallbackTimeoutError } from "dialcache"; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUserWithDeadline", + cacheKey: (userId) => userId, + defaultConfig: DialCacheKeyConfig.enabled(60), + fallbackTimeoutMs: 2_000, + }, +); + +try { + await dialcache.enable(() => getUser("123")); +} catch (error) { + if (error instanceof FallbackTimeoutError) { + logger.warn("source lookup exceeded its DialCache budget", { + useCase: error.useCase, + timeoutMs: error.timeoutMs, + }); + } + throw error; +} +``` + +### When the timer runs + +The timer starts only when the fallback begins: + +- same-key followers share the request-local or process leader's remaining + budget and receive its `FallbackTimeoutError`; +- a remote read failure or timeout starts the fallback timer only when the + source loader begins; +- enabled pass-through invocations where every layer is disabled have + independent timers; +- cache hits create no fallback timer; and +- calls that began outside an enabled context remain true pass-through and are + not timed out, even when the operation has `fallbackTimeoutMs`. + +The fallback deadline does not cover work that happens before fallback. An +active remote read has its own resolved +[remote-read deadline](redis.md#remote-read-deadlines-and-async-liveness), while +a pending config provider or serializer load does not. Serialization and a +Redis write after fallback also remain outside it. Give every injected +operation its own finite, resource-native budget. + +### Event-loop behavior + +Deadline delivery requires the JavaScript event loop to make progress. It +cannot preempt a synchronous fallback prefix or other event-loop blocking. +Rejection can therefore arrive later than the configured duration. + +When control returns, DialCache checks the monotonic deadline before accepting +the result. The timer remains referenced until the fallback settles or times +out. An abandoned enabled fallback can keep an otherwise idle short-lived +process alive until the deadline. + +Shutdown code should drain outstanding DialCache work rather than discarding +its promises. + +### Timeout does not cancel the source + +Timing out: + +1. rejects the DialCache chain; +2. clears its tracked flight normally; +3. ignores a later fallback resolution; and +4. prevents that invocation from proceeding to serializer, Redis, or local + publication. + +The underlying loader is not canceled and may continue its own I/O or side +effects. Give the source operation a native timeout or `AbortSignal` whenever +possible. + +`fallbackTimeoutMs: null` disables the guard and makes finite fallback +settlement entirely application-owned. Use that escape hatch only after +intentionally accepting the liveness risk. + +Timeout failures retain the bounded metrics classification +`error="fallback"` with `in_fallback="true"`. The typed error carries timeout +details without adding high-cardinality labels. + +A shared remote-read timeout emits one `cache_read_timeout` error for the +leader, not one per follower. + +## Inspecting process-scoped flights + +`getCoalescingState()` returns a detached, point-in-time snapshot of +process-scoped flights owned by one `DialCache` instance: + +```ts +const state = dialcache.getCoalescingState(); + +state.process.activeLeaders; +state.process.activeFollowers; +state.process.oldestLeaderAgeMs; // null when idle +``` + +A leader is one exact cache key currently tracked by the instance-scoped +coalescer. A follower is each later invocation that joined that pending leader; +the initiating invocation is not counted as a follower. + +Followers remain counted until their leader settles because abandoning a +JavaScript promise is not observable. Request-local flights are deliberately +excluded because their lifecycle is bounded by the outer `enable()` scope. +`oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is +requested. + +## Admission control remains application-owned + +There is no library-wide flight cap or age-based replacement. + +A registry cap would bound only DialCache metadata. Overflow or eviction could +still create unbounded source work and unsafe duplicate publication. Finite +operation deadlines provide eventual cleanup; application admission control +and backpressure remain responsible for bounding simultaneous distinct-key +work. + +Monitor: + +- active leader count; +- active follower count; +- oldest leader age; +- remote-read timeout errors; +- fallback deadline errors; and +- source concurrency and saturation. + +Use those signals to verify that application budgets and admission control hold +under production load. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..fbaf568 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,434 @@ +# Configuration and cache layers + +[Back to the README](../README.md) + +This guide covers reusable cached functions, one-shot inline loaders, cache +identity, runtime policy, request-local and process-local behavior, and +cached-value ownership. For the shared remote layer, see +[Redis and Valkey](redis.md). + +## Defining cache operations + +### Reusable cached functions + +`cached(fn, options)` wraps a function; the wrapped callable has the same +parameters and always returns a `Promise`. + +| Option | Required | Description | +| --- | --- | --- | +| `keyType` | yes | The kind of id the key addresses, such as `"user_id"`. Together with the id, this is the invalidation unit for tracked entries. | +| `useCase` | yes | Identifies the individual cache. It is part of the stored key and a metrics label. | +| `cacheKey` | yes | Selects a bare id or `{ id, args }` from `fn`'s parameters. | +| `defaultConfig` | no | Provides the `DialCacheKeyConfig` baseline that runtime config overlays field by field. | +| `serializer` | when the return type is not statically JSON-compatible | Selects a per-function `Serializer` for Redis values; see [Serialization](redis.md#serialization). | +| `trackForInvalidation` | no; default `false` | Opts this use case's Redis entries into watermark-based [targeted invalidation](invalidation.md). | +| `fallbackTimeoutMs` | no; default `60_000` | Sets the fallback deadline in milliseconds, up to 2,147,483,647. `null` disables it; see [Fallback deadlines](coalescing.md#fallback-deadlines). | + +`useCase` is validated when the function is registered. A duplicate within one +`DialCache` instance throws `UseCaseIsAlreadyRegisteredError`, and the internal +name `watermark` throws `UseCaseNameIsReservedError`. + +### One-shot inline loaders + +`getOrLoad(load, options)` runs one zero-argument synchronous or asynchronous +loader through the same cache layers, runtime policy, coalescing, invalidation, +metrics, serialization, and deadline behavior as `cached()`. Cache-plumbing +failures fall through to the loader; loader failures still reject and clear +their tracked flight: + +```ts +const profile = await dialcache.enable(() => + dialcache.getOrLoad( + async () => { + const user = await db.getUser(userId); + return renderProfile(user, locale); + }, + { + keyType: "user_id", + useCase: "BuildProfile", + key: { id: userId, args: { locale } }, + defaultConfig: DialCacheKeyConfig.enabled(60), + }, + ), +); +``` + +The options match `cached()` except that the direct `key` replaces the +`cacheKey` selector. `defaultConfig` and `fallbackTimeoutMs` are validated and +snapshotted for each invocation. Outside an enabled scope, DialCache calls +`load` directly without constructing a key or resolving runtime policy. + +`getOrLoad()` does not register its `useCase` or detect duplicates, but it still +rejects the reserved internal name `"watermark"`. + +Repeated calls should reuse one stable, deployment-defined name such as +`"BuildProfile"`. Never derive it from a user, request, id, or other +high-cardinality input because it is part of both cache identity and metrics +labels. Put those values in `key` instead. + +Every captured value that can change the result belongs in the bare id or +`{ id, args }` key. Concurrent same-key calls may share one caller's in-flight +loader and cached value, so all call sites for that identity must also agree on +value meaning and serialization. + +Prefer `cached()` for reusable loaders and `getOrLoad()` for calculations +intentionally local to one call site. + +## Keys, ids, and extra dimensions + +For `cached()`, the required `cacheKey` selector receives the wrapped +function's inferred parameters. `getOrLoad()` accepts the same bare id or +`{ id, args }` shape directly through `key`: + +```ts +const searchPosts = dialcache.cached( + (userId: string, page: number, filter: string) => + db.searchPosts(userId, page, filter), + { + keyType: "user_id", + useCase: "SearchPosts", + cacheKey: (userId, page, filter) => ({ + id: userId, + args: { page, filter }, + }), + defaultConfig: DialCacheKeyConfig.enabled(60), + }, +); + +await dialcache.enable(() => searchPosts("u1", 2, "active")); +``` + +The selected or direct key is the value-identity contract. It must include +every input dimension that can affect the returned value. Otherwise, distinct +calls can reuse the same cached value or share the same in-flight fallback +through request coalescing. + +### Namespace + +`DialCacheConfig.namespace` is the logical cache namespace and the first +component of every key. It defaults to `"urn"`, producing keys such as +`urn:user_id:123#GetUser`. + +Set a stable application-specific value when applications or environments may +share one Redis deployment: + +```ts +const dialcache = new DialCache({ + namespace: "production-users-api", + redis: { client: dialCacheRedisClient }, +}); +``` + +That produces Redis keys beginning with `production-users-api:...`, or +`{production-users-api:...}` for invalidation-tracked values. `namespace` is +DialCache's single cache-identity and key-partitioning setting. It participates +in request-local, process-local, Redis, coalescing, deterministic ramp, +invalidation, and metrics. + +A namespace may not contain `{` or `}` because DialCache reserves those +characters for Redis Cluster hash tags. + +### Identity rules + +- **`keyType` plus `id` is the invalidation unit for tracked Redis entries.** + `dialcache.invalidateRemote("user_id", "123", futureBufferMs)` writes one + watermark for that user. Any tracked Redis entry with the same `keyType` and + `id` is refreshed across all `args` variants when Redis is read. Untracked + entries do not consult the watermark. Invalidation does not evict existing + request-local or process-local entries. +- **`args` are part of the cache key.** Different arguments produce different + entries, but targeted invalidation is by id rather than by argument. +- **Scalar equality is string-based.** For matching surrounding dimensions: + - numeric `1`, string `"1"`, and bigint `1n` identify the same key; and + - argument values `null` and `"null"` match, `-0` matches `0`, and an + `undefined` argument is omitted. + + If a deployment changes the logical meaning represented by a scalar, change + an explicit identity dimension such as `keyType`, `useCase`, or an argument + name or value. +- **Non-key inputs still reach the loader.** A database handle can be a normal + function parameter ignored by `cacheKey` or a value captured by a + `getOrLoad()` loader. Concurrent same-key misses share the leader's + execution. Do not omit values such as `AbortSignal`, auth context, locale, or + other request-scoped inputs unless sharing one result is correct. +- **Methods need a receiver.** Pass `obj.method.bind(obj)` or + `(...args) => obj.method(...args)`; a bare `obj.method` reference loses + `this`. + +### Changing a namespace + +Changing `namespace` intentionally creates a cold-cache boundary across every +layer. Old and new keyspaces do not share Redis values or invalidation +watermarks. + +During an overlapping deployment, an invalidation handled by one version is +invisible to the other. The other version can continue serving a stale tracked +value until its value TTL expires. If remote invalidation correctness matters, +a normal rolling namespace change is unsafe. + +Use a coordinated no-overlap cutover, or an operational bridge that prevents +both versions from serving remote cache across mutations. For example, +temporarily disable and clear remote caching during the transition. After the +cutover, provision for fallback and refill load, and allow old Redis keys to +expire by TTL. + +## Runtime config and ramp controls + +Instance-wide behavior is set through the `DialCache` constructor: + +| `DialCacheConfig` option | Default | Description | +| --- | --- | --- | +| `namespace` | `"urn"` | Logical cache namespace and first key component. | +| `redis` | none | `{ client, readTimeoutMs?, serializer? }`; enables the [remote layer](redis.md). Remote reads default to a 50 ms deadline. | +| `localMaxSize` | `10_000` | Global process-local entry cap. `0` disables process-local storage. Must be a nonnegative safe integer. | +| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the operation's `defaultConfig`; `null` applies no overrides. | +| `metrics` | disabled | A `DialCacheMetricsAdapter`; see [Observability](observability.md). | +| `logger` | `console` | Receives operational cache failures through `debug`, `warn`, and `error`. | + +Per-invocation policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` +maps keyed by `CacheLayer.LOCAL` and `CacheLayer.REMOTE`, a `requestLocal` +boolean, and an optional `remoteReadTimeoutMs`. + +### Baseline and overlay precedence + +Every cached definition or `getOrLoad()` invocation can provide an optional +per-use-case `defaultConfig`. That is the baseline policy. The +`cacheConfigProvider` result is a sparse field-level overlay on it. + +Enablement fields use this precedence: + +```text +runtime field -> defaultConfig field -> DialCache disabled baseline +``` + +The disabled baseline sets `requestLocal` to `false` and leaves the +process-local and remote TTLs unset. A shared layer with no effective TTL is +disabled by policy. A shared layer with an effective TTL but no effective ramp +defaults to a 100% ramp. + +The remote-read deadline has two additional fallbacks: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +This value bounds how long DialCache waits for an active Redis or Valkey read. +It can be tuned per use case at runtime, but it cannot be disabled. + +`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined`, so the +overlay can distinguish omission from an explicit `false`. Its effective value +still defaults to `false` after resolution. + +A provider result of `null`, or a defensive `undefined`, applies no overrides. +An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the +baseline. + +Use explicit values to replace inherited policy: + +- `requestLocal: false` disables request-local caching; +- a shared-layer ramp of `0` disables that layer; and +- `DialCacheKeyConfig.disabled()` turns request-local off and ramps both shared + layers to `0`. + +### Validation and snapshots + +DialCache validates `defaultConfig` when `cached()` registers a definition and +whenever `getOrLoad()` is invoked: + +- TTLs must be positive safe integers; +- ramps must be finite percentages from 0 to 100; +- layer maps must be objects; +- `requestLocal` must be a boolean when present; and +- remote-read deadlines must be positive safe integers no greater than + 2,147,483,647 milliseconds. + +Invalid instance `redis.readTimeoutMs` values throw during `DialCache` +construction. Invalid defaults are rejected when `cached()` registers a +definition or `getOrLoad()` is invoked. `null`, zero, fractional, non-finite, +string, and larger timeout values are invalid; remote reads have no unbounded +escape hatch. + +Each registration or one-shot invocation captures an immutable internal +snapshot, so mutating the supplied config or its maps later does not change +that operation's baseline. Runtime policy changes belong in the provider's +returned overlay. + +Runtime TTL and ramp leaves are used as supplied rather than falling back to +valid default leaves: + +- an invalid TTL disables that layer with `invalid_ttl`; +- a nonnumeric or non-finite ramp disables it with `invalid_ramp`; +- a finite runtime ramp retains a defensive clamp to 0 through 100; and +- other valid layers can continue to run. + +Invalid leaves also record a `config_resolution` error, distinguishing provider +garbage from an intentional ramp-down. A malformed runtime config object, +layer-map shape, `requestLocal`, or `remoteReadTimeoutMs` value fails config +resolution for the whole invocation. DialCache records `config_resolution`, +marks the no-layer path `config_error`, and runs the fallback without a Redis +read or write. + +### Provider behavior + +`cacheConfigProvider` is called for every enabled cache invocation before any +cache lookup. Keep it cheap, cache remote or config-store reads inside the +provider, and give asynchronous work a finite application-owned deadline. + +DialCache fetches and resolves one config snapshot per enabled invocation. +Provider errors do not activate defaults: they fail open, record +`config_error`, and execute the fallback uncached. + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + readTimeoutMs: 75, + }, + cacheConfigProvider: async (key) => { + if (key.useCase === "GetUser") { + return new DialCacheKeyConfig({ + // Sparse override: inherit both TTLs and the local ramp. + ramp: { [CacheLayer.REMOTE]: 25 }, + // Per-use-case override of the instance's 75 ms read deadline. + remoteReadTimeoutMs: 35, + }); + } + return null; + }, +}); + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + // Omitted ramps default to 100% because these layers have TTLs. + ttlSec: { + [CacheLayer.LOCAL]: 30, + [CacheLayer.REMOTE]: 300, + }, + }), + }, +); +``` + +Ramp values are thresholds from 0 to 100. `0` disables the layer, `100` enables +it for every key, and an intermediate value selects keys whose DialCache-owned +deterministic bucket for the full cache key and layer is below that threshold. + +For a fixed cache identity and layer, increasing a ramp only adds keys and +decreasing it only removes keys; it does not reshuffle existing membership. +Local and remote cohorts are layer-specific. + +Ramps select key cohorts, not requests or load, so a ramp of `10` does not +guarantee 10% of calls, especially for a small or skewed key population. +DialCache keeps the assignment stable across releases. + +Applications that need an externally coordinated cohort can use +`cacheConfigProvider` to return a sparse per-key ramp override of `0` or `100`. + +Ramping down bypasses affected entries; it does not evict them, so a later +ramp-up can reuse entries that remain valid. + +## Request-local cache + +Set `requestLocal: true` to memoize resolved values for the lifetime of the +outermost `enable()` scope: + +```ts +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ requestLocal: true }), + }, +); +``` + +`requestLocal` is a runtime boolean rather than a TTL/ramp-controlled +`CacheLayer`. The provider can turn it on or off for each invocation. +`DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and remote +caching, so request-local caching must be selected explicitly. + +The resolved config applies to the whole invocation. When `requestLocal` is +false, the invocation skips request-local lookup and storage without deleting a +value already memoized in the scope. A later invocation that enables it can +reuse that value. + +The outermost `enable()` call owns the request-local lifetime; nested `enable()` +calls reuse the same scope. State is allocated lazily, so scopes that use only +process-local or remote caching do not allocate it. + +Wrap the complete Node HTTP handler so the scope matches the request: + +```ts +import { createServer } from "node:http"; + +const server = createServer((req, res) => { + void dialcache + .enable(async () => { + const user = await getUser(readUserId(req)); + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(user)); + }) + .catch((error: unknown) => handleRequestError(error, res)); +}); +``` + +Request-local storage has no capacity limit, eviction, or overflow mode. Values +are retained until the outermost callback settles. Use it for short-lived +scopes with bounded key cardinality. Split long-running streams or large batch +jobs into smaller scopes. + +## Process-local cache + +The process-local layer, `CacheLayer.LOCAL`, uses one LRU per `DialCache` +instance. It keeps at most 10,000 entries by default across all use cases while +retaining each entry's configured TTL. + +Set `localMaxSize` to a nonnegative safe integer to change the global entry cap. +`0` disables process-local storage: + +```ts +const dialcache = new DialCache({ localMaxSize: 25_000 }); +``` + +The limit counts entries rather than estimating JavaScript object memory. +Recently read entries stay resident ahead of less recently used entries when +the limit is reached. + +## Cached-value ownership + +Treat values returned by cached functions or `getOrLoad()` as immutable. +DialCache does not clone or freeze values stored in request-local or +process-local memory. +Mutating a cached object can be observed by: + +- later callers in the same request; +- callers in other requests that hit the process-local cache; and +- callers that coalesced onto the same in-flight result. + +This contract includes nested objects and arrays, `Map`, `Set`, `Buffer`, typed +arrays, and class instances. Redis deserialization can produce a different +reference from an in-memory hit, so reference identity is layer-dependent and +is not part of the API contract. + +Copy a value explicitly before changing it: + +```ts +const sharedUser = await getUser("123"); +const editableUser = structuredClone(sharedUser); +editableUser.displayName = "New name"; +``` + +Use a narrower copy when its semantics are sufficient. The ownership boundary +remains the caller's responsibility. diff --git a/docs/invalidation.md b/docs/invalidation.md new file mode 100644 index 0000000..b853ec5 --- /dev/null +++ b/docs/invalidation.md @@ -0,0 +1,205 @@ +# Targeted invalidation + +[Back to the README](../README.md) + +DialCache can invalidate related Redis entries without scanning or enumerating +keys. The mechanism is opt-in, remote-only, and based on per-identity Redis +watermarks. + +Read this complete contract before using targeted invalidation for mutable +production data. Correctness depends on cache-layer policy, Redis clock +synchronization, and an application-owned timing buffer. + +## Configure a tracked use case + +Set `trackForInvalidation: true` on a Redis-backed cached function or +`getOrLoad()` operation. After the source mutation commits, call +`dialcache.invalidateRemote(keyType, id, futureBufferMs)`: + +```ts +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: dialCacheRedisClient }, +}); + +// Chosen from this application's clock-skew bound and measured +// worst-case source and fallback timings. +const USER_INVALIDATION_BUFFER_MS = 5_000; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetMutableUser", + cacheKey: (userId) => userId, + trackForInvalidation: true, + // Strongly invalidated mutable data should not use in-memory layers. + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { + [CacheLayer.REMOTE]: 300, + }, + ramp: { + [CacheLayer.REMOTE]: 100, + }, + }), + }, +); + +await updateUser("123", patch); +await dialcache.invalidateRemote( + "user_id", + "123", + USER_INVALIDATION_BUFFER_MS, +); +``` + +The buffer is an application-owned safety value. DialCache cannot choose a +universally safe nonzero default. + +## Identity and Redis Cluster placement + +Invalidation writes a watermark at: + +```text +{encodedNamespace:encodedKeyType:encodedId}#watermark +``` + +Tracked Redis values use the same Redis Cluster hash tag. For example: + +```text +{users-api:user_id:123}?locale=en#GetMutableUser:dialcache-frame-v1 +``` + +The value and watermark therefore live in the same Redis Cluster slot. Key +components are percent-encoded before joining, so delimiters inside ids or +arguments cannot collide with delimiters in the key format. + +`namespace` may never contain `{` or `}`; tracked `keyType` and `id` values may +not contain them because those three components form the hash tag. `args` and +`useCase` are encoded outside the hash tag and may contain braces. + +The internal `:dialcache-frame-v1` suffix identifies values written with +DialCache's binary protocol. Watermarks are stored as decimal timestamps. + +`keyType` plus `id` is the invalidation unit. One watermark covers every tracked +`useCase` and `args` variant with that identity. Untracked values do not consult +it. + +## Read and write behavior + +A tracked Redis value whose Redis-created timestamp is older than or equal to +the watermark is treated as stale and refreshed through fallback. + +`invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the +greater of: + +- its existing value; and +- Redis's current time plus the buffer. + +While that future window is active: + +1. A tracked Redis read treats the covered value as a miss. +2. The invocation runs its fallback. +3. If that fallback reaches the tracked Redis write before the window ends, + Redis rejects the write. +4. DialCache also suppresses the corresponding process-local population. +5. The fallback value still returns to its caller. + +Request-local memoization remains unconditional. An invocation whose remote +layer is disabled or ramped out does not consult the watermark and is not +fenced by it. + +This is a timing contract, not a cancellation or acquisition fence. The buffer +blocks stale fallback results from passing the tracked Redis write only while +the configured window remains active. It does not cancel the fallback or force +it to read from an authoritative source. + +If a tracked remote read rejects or exceeds its deadline, DialCache cannot +establish watermark safety. It runs the fallback but skips both the Redis write +and process-local publication. This differs from a normal tracked miss, which +can attempt the fenced Redis write. Untracked fallbacks may still populate +process-local cache, and request-local memoization remains unconditional. + +## Redis clock contract + +The bundled timestamp protocol assumes synchronized system clocks across every +Redis node eligible for primary promotion. + +Redis does not guarantee that `TIME` is monotonic across nodes, and DialCache +does not detect or compensate for cross-node clock skew. If the assumption is +violated, failover can: + +- temporarily suppress tracked cache fills; or +- allow a pre-invalidation value to remain readable until it expires or a later + invalidation advances the watermark past its timestamp. + +Monitor and bound the maximum negative clock skew across all promotion-eligible +nodes. Include that bound when sizing `futureBufferMs`. + +## Watermark durability + +Watermarks are invalidation state, not disposable cache entries. Redis must +preserve each marker for its derived TTL with `noeviction` or an equivalent +guarantee. Choose persistence, restore, and failover behavior that matches the +application's consistency requirements. + +Losing a marker through eviction, failover, restore, or external deletion +removes its prior publication fence. A missing marker makes tracked reads miss, +but a later tracked write creates a new baseline and can publish data that a +lost future watermark would have rejected. + +Redis replication is asynchronous. DialCache does not issue `WAIT` and does not +provide strong consistency across failover. + +## Watermark lifetime + +Tracked writes create a missing baseline watermark and ensure its TTL is at +least the value TTL plus one minute. They never shorten a longer or persistent +watermark TTL. + +Invalidation ensures the TTL covers both the requested future buffer and any +still-future existing watermark, plus one minute. It also preserves a longer or +persistent TTL. There is no fixed or configurable retention floor, and reads do +not extend watermark lifetime. + +## Choosing `futureBufferMs` + +`futureBufferMs` must be a nonnegative safe integer. The API default is zero, +but zero provides no stale-publication protection once Redis time advances. + +Every production invalidation should pass a named, application-owned nonzero +value based on measured or conservatively bounded timings. Size it to cover: + +- maximum expected negative clock skew between promotion-eligible Redis nodes; +- source visibility or replication lag; +- the full remaining tail of any fallback that may already have observed the + pre-mutation value; +- `serializer.dump`; +- Redis client queue and network latency; +- Lua script execution; +- the Redis write itself; and +- a safety margin. + +Invalidate only after the source mutation commits. + +Underestimating the interval can allow a delayed stale fallback to repopulate +Redis after the watermark window ends. Overestimating it lengthens the tracked +Redis miss and write-suppression window, increasing fallback load without +publishing stale values. + +A larger buffer does not delay or suppress returning fallback values to +callers. + +## In-memory layers remain local + +Targeted invalidation is remote-only. `invalidateRemote` does not evict existing +request-local or process-local entries. + +Strongly invalidated mutable data should disable request-local and process-local +caching. A short process-local TTL is appropriate only when the application +explicitly accepts that bounded stale-read window. + +If those layers remain enabled, their existing values can be returned without +reaching the remote watermark. diff --git a/docs/maintainers.md b/docs/maintainers.md new file mode 100644 index 0000000..4da5075 --- /dev/null +++ b/docs/maintainers.md @@ -0,0 +1,76 @@ +# Maintainer guide + +[Back to the README](../README.md) + +## Cache-path benchmark + +From a repository checkout, install dependencies and run: + +```bash +corepack pnpm benchmark:request-local +``` + +The command builds `dist` before reporting six scenarios: + +- sequential request-local hits; +- sequential process-local hits; +- enabled bounded fallbacks; +- request-local coalescing fan-out; +- process coalescing fan-out; and +- Redis read-deadline coalescing. + +The benchmark is a maintainer tool and is not included in the published +package. It asserts fallback counts, coalescing state, returned values, and one +semantic read and one cleaned-up deadline timer for the remote coalescing +scenario. It deliberately applies no timing threshold. + +Override its work sizes with: + +- `DIALCACHE_BENCH_ITERATIONS`; and +- `DIALCACHE_BENCH_FANOUT`. + +## Releasing + +Publishing starts by manually running the `Release` workflow from current +`main`. + +After the package checks pass, Semantic Release selects the next version from +Conventional Commits since the highest stable `vX.Y.Z` tag: + +- breaking changes bump major; +- `feat` bumps minor; and +- every other normal PR-title type bumps patch. + +Patch types are `fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, +`chore`, `ci`, and `revert`. The highest required bump wins. + +The workflow opens a `release: ` pull request whose only change is the +matching `package.json` version. `release` is a reserved Conventional Commit +type configured not to request another release, so the version-control commit +does not cause an extra bump. + +GitHub marks workflow runs for a pull request opened with `GITHUB_TOKEN` as +approval-required. Approve those runs, review the pull request, and squash-merge +it normally through the protected branch. + +The merge triggers the publish job. Before any release side effect, it verifies: + +- current `main`; +- the release commit subject; +- the one-file diff; +- the package version; +- the absent tag; and +- Semantic Release's independently calculated version and commit. + +It then reruns the package checks and asks Semantic Release to: + +1. create the matching Git tag; +2. publish the public npm package with provenance; and +3. publish the GitHub release. + +The repository must enable **Allow GitHub Actions to create and approve pull +requests** under Actions workflow permissions. + +The workflow uses that capability only to create the version pull request. It +never approves or merges one, and no ruleset bypass actor or persistent release +credential is required. diff --git a/docs/observability.md b/docs/observability.md new file mode 100644 index 0000000..ac08f1c --- /dev/null +++ b/docs/observability.md @@ -0,0 +1,261 @@ +# Observability + +[Back to the README](../README.md) + +Metrics are disabled unless a `DialCacheMetricsAdapter` is passed to the +constructor. `new DialCache()` does not import a metrics backend, register +collectors, or emit metrics. + +DialCache provides first-party adapters for Prometheus and Datadog. Both use +caller-created, caller-owned clients and preserve one backend-neutral set of +bounded labels. + +## Prometheus + +Install `prom-client` separately: + +```bash +pnpm add prom-client@^15.1.3 +``` + +Create the registry your application owns, then pass an explicit adapter to +DialCache: + +```ts +import { Registry } from "prom-client"; +import { DialCache } from "dialcache"; +import { createPrometheusDialCacheMetrics } from "dialcache/prometheus"; + +const registry = new Registry(); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createPrometheusDialCacheMetrics({ + registry, + prefix: "myapp_", + }), +}); + +app.get("/metrics", async (_req, res) => { + res.type(registry.contentType).send(await registry.metrics()); +}); +``` + +The adapter requires a caller-owned `Registry`. It never uses the global +default registry, and it does not clear or otherwise own the registry +lifecycle. + +Multiple adapters with the same registry and prefix reuse existing collectors +when their type, help, labels, histogram buckets, and exemplar mode match. +Adapter construction fails before registering anything if a same-name +collector has an incompatible schema. Use a unique prefix or separate registry +to resolve a collision. + +### Prometheus metrics + +The names below exclude the optional caller-selected prefix: + +| Metric | Type | Labels | Description | +| --- | --- | --- | --- | +| `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by request-local or process scope | +| `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | + +The disabled reasons are: + +- `context`; +- `policy_disabled`; +- `invalid_ttl`; +- `invalid_ramp`; +- `ramped_down`; and +- `config_error`. + +`policy_disabled` means that a process-local or remote layer has no effective +TTL after runtime overlays. This is an intentional policy result, including the +default when `defaultConfig` is omitted, rather than a configuration-loading +failure. + +Every metric includes `cache_namespace`, even disabled-context, +key-construction, coalescing, and invalidation paths that do not have a +constructed key. Its value is `DialCacheConfig.namespace`, which defaults to +`urn`. + +The `layer` label is: + +- `request_local`; +- `local`, meaning process-local; +- `remote`; or +- `noop` for disabled-context, key-construction, and config-provider failures + where no cache layer was reached. + +The bounded `scope` label on `dialcache_coalesced_counter` distinguishes +`request_local` from `process`. `scope="process"` coordinates calls only within +one `DialCache` instance; separate instances in the same process do not share +in-flight state. + +## Datadog + +Install `hot-shots` separately: + +```bash +pnpm add hot-shots@^17.0.0 +``` + +Create the DogStatsD client your application owns, then pass it to the Datadog +adapter: + +```ts +import StatsD from "hot-shots"; +import { DialCache } from "dialcache"; +import { createDatadogDialCacheMetrics } from "dialcache/datadog"; + +const dogStatsD = new StatsD({ + host: process.env.DD_AGENT_HOST, + globalTags: { + service: "users-api", + env: process.env.DD_ENV ?? "development", + }, + errorHandler: (error) => + logger.warn("DogStatsD error", { error }), +}); + +const dialcache = new DialCache({ + namespace: "users-api", + metrics: createDatadogDialCacheMetrics({ + client: dogStatsD, + observationMetricType: "distribution", + namespace: "dialcache", + }), +}); + +// Drain outstanding cache operations before application shutdown. +dogStatsD.close(); +``` + +`hot-shots` is the supported and tested client, but the adapter depends only on +the exported `DatadogDogStatsDClient` structural interface. + +DialCache does not: + +- import or install `hot-shots`; +- create a client; +- flush buffers; +- close sockets; or +- otherwise own the client lifecycle. + +### Distribution or histogram + +`observationMetricType` is required. + +Choose `"distribution"` when latency and size percentiles must aggregate across +hosts. Enable the desired distribution percentiles and aggregations in +Datadog. + +Choose `"histogram"` when host-level histogram aggregation matches the existing +Datadog setup. The choice applies uniformly to all four duration and size +metrics. Both modes produce Datadog custom metrics. + +Distribution volume scales with unique tag-value combinations. Datadog counts +five baseline aggregations per combination; enabling percentile aggregations +adds five more. Review +[Datadog's custom-metrics billing guidance](https://docs.datadoghq.com/account_management/billing/custom_metrics/) +before rollout. + +Do not send both observation types under the same metric namespace. When +changing types, use a new namespace during migration so one metric identity +never mixes histogram and distribution points. + +### Datadog namespaces + +`DatadogMetricsOptions.namespace` is the metric-name namespace and defaults to +`dialcache`. It is separate from `DialCacheConfig.namespace`, the logical cache +namespace emitted as the `cache_namespace` tag. + +The Datadog metric namespace must: + +- start with a letter; +- contain only letters, numbers, underscores, and dot-separated non-empty + segments; and +- produce final metric names no longer than 200 characters. + +The adapter rejects invalid namespaces and overlong final names instead of +relying on client-side normalization. + +A `hot-shots` `prefix` is applied after the adapter constructs the name. Include +that prefix when checking final length, and avoid accidentally combining it +with the adapter namespace. Client-level `globalTags` are appended by +`hot-shots`; the table below lists only tags added by DialCache. + +### Datadog metrics + +The adapter emits exact increments of `1` for counters and preserves seconds +and bytes without unit conversion: + +| Metric | Type | Tags | Description | +| --- | --- | --- | --- | +| `dialcache.request.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | +| `dialcache.miss.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | +| `dialcache.disabled.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips by bounded reason | +| `dialcache.error.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache or fallback errors by bounded failure site | +| `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | +| `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | +| `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | +| `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | +| `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | +| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | + +Synchronous client throws are isolated by DialCache's fail-open metrics +boundary. Buffered transport failures happen outside that synchronous call. +Configure the DogStatsD client's error handling and shutdown behavior as part +of application ownership. + +## Error categories + +The `error` label reports the operation that failed instead of copying the +thrown value's class or `Error.name`: + +| `error` | Meaning | +| --- | --- | +| `key_construction` | The cache-key selector or `DialCacheKey` construction failed | +| `config_resolution` | Runtime or layer configuration validation or resolution failed | +| `cache_read` | A process-local read or non-timeout remote read failed | +| `cache_read_timeout` | A remote read exceeded its effective DialCache deadline | +| `cache_write` | A process-local or remote cache write failed | +| `serialization_load` | Deserializing a Redis payload failed | +| `serialization_dump` | Serializing a value for Redis failed | +| `invalidation` | Writing an invalidation watermark failed | +| `fallback` | The source loader failed or exceeded its DialCache deadline | +| `unknown` | Reserved for a future failure site that cannot be classified otherwise | + +These values are defined by the backend-neutral core and are identical for +every adapter. + +Remote-read timeouts use `layer="remote"` and `in_fallback="false"`. They are +errors rather than misses, and the remote get-duration observation includes +the wait. Coalesced followers do not multiply the timeout error. Deadline +details remain out of labels and are available on the logged +`RedisReadTimeoutError`. + +Raw thrown values, error names, messages, cache ids, arguments, and Redis keys +are never included in labels. Operational errors still reach the configured +logger. `in_fallback` remains the explicit distinction between cache plumbing +and application fallback failures. + +## Custom adapters + +Implement `DialCacheMetricsAdapter` and pass it through +`new DialCache({ metrics })` for another telemetry backend. + +Every backend-neutral label object exposes the logical namespace as camel-case +`cacheNamespace`. Map it to the backend's `cache_namespace` label or tag. This +field is present even when no key or cache layer was reached. + +Synchronous adapter failures are isolated from cache behavior and application +fallbacks. Omit `metrics` to disable metrics entirely. diff --git a/docs/redis.md b/docs/redis.md new file mode 100644 index 0000000..bd3f81d --- /dev/null +++ b/docs/redis.md @@ -0,0 +1,377 @@ +# Redis and Valkey + +[Back to the README](../README.md) + +DialCache's remote TTL layer supports standalone Redis, standalone Valkey, and +Redis Cluster. The application creates, connects, configures, drains, and closes +the underlying client. DialCache borrows a semantic `DialCacheRedisClient` and +does not own the connection lifecycle. + +## Install a client + +Choose one supported integration: + +```bash +# node-redis +pnpm add redis@~4.7.1 + +# or Valkey GLIDE +pnpm add @valkey/valkey-glide +``` + +## node-redis + +Register DialCache's native scripts when creating the client, connect it, and +pass the semantic adapter to `DialCache`: + +```ts +import { createClient } from "redis"; +import { DialCache } from "dialcache"; +import { + createNodeRedisDialCacheClient, + dialcacheRedisScripts, +} from "dialcache/node-redis"; + +const redisClient = createClient({ + url: process.env.REDIS_URL, + scripts: dialcacheRedisScripts, + disableOfflineQueue: true, + commandsQueueMaxLength: 1_000, + socket: { connectTimeout: 2_000 }, +}); + +await redisClient.connect(); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { + client: createNodeRedisDialCacheClient(redisClient), + }, +}); + +async function shutdown(): Promise { + // Stop new work and await every cached call and invalidation first. + await redisClient.quit(); +} +``` + +`redis.client` is required when the remote layer is configured. Node-redis +users should register the supplied scripts and wrap the connected client with +`createNodeRedisDialCacheClient` as shown above. Active remote reads have a +50-millisecond DialCache deadline by default. Set `redis.readTimeoutMs` for an +instance-wide value or use `DialCacheKeyConfig.remoteReadTimeoutMs` for +per-use-case static and runtime policy. + +The adapter computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` +after `NOSCRIPT`. Its cluster client routes scripts by their first key and +performs that fallback on the selected shard. Tracked reads are deliberately +routed to primaries so a lagging replica cannot hide an invalidation watermark. + +Deployments using tracked invalidation must also satisfy the +[watermark durability](invalidation.md#watermark-durability) contract. + +## Valkey GLIDE + +Pass an already-created standalone or cluster client and the exact module +namespace that created it: + +```ts +import * as valkeyGlide from "@valkey/valkey-glide"; +import { DialCache } from "dialcache"; +import { createValkeyGlideDialCacheClient } from "dialcache/valkey-glide"; + +const glideClient = await valkeyGlide.GlideClient.createClient({ + addresses: [{ host: "127.0.0.1", port: 6379 }], + requestTimeout: 2_000, + advancedConfiguration: { + connectionTimeout: 2_000, + }, +}); + +const redisClient = createValkeyGlideDialCacheClient( + glideClient, + valkeyGlide, +); + +const dialcache = new DialCache({ + namespace: "users-api", + redis: { client: redisClient }, +}); + +function shutdown(): void { + // Drain cached calls and invalidations before releasing resources. + redisClient.dispose(); + glideClient.close(); +} +``` + +DialCache uses the supplied namespace's `Script` constructor and +`Decoder.Bytes` value without importing a GLIDE runtime itself. Passing the same +module namespace that created the client prevents linked workspaces or +applications with another installed GLIDE version from mixing native script +handles. + +The GLIDE adapter uses GLIDE's native script lifecycle and byte decoder. GLIDE +routes scripts from their declared keys. + +## Lifecycle ownership + +The application owns the complete Redis lifecycle: + +1. Create and connect the underlying client. +2. Construct the semantic DialCache adapter. +3. Pass that adapter as `redis.client`. +4. During shutdown, stop starting DialCache-backed work. +5. Await every outstanding cached-function, `getOrLoad()`, and + `invalidateRemote()` promise, including fallbacks that may still write + Redis. +6. Drain or terminate client-native Redis work that may have outlived + DialCache's remote-read wait. +7. Dispose adapter-owned resources. +8. Close the underlying connection. + +DialCache has no `close()` or drain method. It never disposes or closes caller +resources. + +The node-redis adapter owns no additional resources, so close the underlying +client after draining work. + +The GLIDE adapter owns five native `Script` handles but not the wrapped +connection. Call its idempotent `dispose()` after operations finish and before +closing GLIDE. Disposing while an adapter operation is in flight throws rather +than releasing a live script. A DialCache read timeout does not prove that the +client-side invocation has settled. + +## Remote-read deadlines and async liveness + +Every active semantic remote-read leader has a finite monotonic deadline. +DialCache uses this precedence for `cached()` and `getOrLoad()`: + +```text +runtime remoteReadTimeoutMs + -> defaultConfig.remoteReadTimeoutMs + -> redis.readTimeoutMs + -> 50 ms +``` + +Each explicit value must be a positive safe integer no greater than +2,147,483,647 milliseconds. Remote reads have no unbounded escape hatch. +Outside an enabled scope, on a local hit, or when remote policy is disabled or +ramped out, DialCache creates no remote-read timer. + +### Timeout and fail-open behavior + +When the deadline expires, DialCache: + +1. aborts the optional adapter signal; +2. logs a root-exported `RedisReadTimeoutError` carrying `useCase` and + `timeoutMs`; +3. records `cache_read_timeout`; +4. consumes and ignores any late read fulfillment or rejection; and +5. runs the source fallback. + +The deadline bounds caller wait and cache publication. It does not guarantee +server-side cancellation, and an event-loop-blocking operation cannot be +preempted. When control returns, DialCache still checks the monotonic deadline +before accepting the result. + +A remote read rejection or timeout does not count as a miss and never triggers +a second Redis operation. After fallback, an untracked key may still populate +an active process-local cache. A tracked key suppresses process-local +publication because watermark safety was not established. Request-local +memoization remains unconditional. + +Same-key callers in one request-local or process coalescing scope share the +leader's read, timer, and remaining budget. A later independent invocation may +start a new remote read even if the prior client operation is still settling. + +The `fallbackTimeoutMs` timer is separate and starts only if and when the source +loader begins. The remote-read timer covers neither config resolution, +serializer loading, the fallback, Redis writes, nor invalidation. + +### Custom-client read context + +The semantic boundary exposes an optional second argument: + +```ts +interface RedisReadContext { + readonly timeoutMs: number; + readonly signal: AbortSignal; +} + +interface DialCacheRedisClient { + read( + request: RedisReadRequest, + context?: RedisReadContext, + ): Awaitable; +} +``` + +The optional argument keeps existing one-argument custom clients structurally +compatible. Adapters should use the signal for cooperative cancellation where +their client supports it, but the core deadline remains authoritative when +they do not. + +The bundled node-redis adapter forwards the signal in per-command options. This +can remove queued work where supported, but aborting after dispatch cannot +unsend a command or prove that Redis stopped executing it. + +The GLIDE script API has no per-invocation signal, so a timed-out script +invocation may continue inside the adapter. Its configured +[`requestTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.BaseClientConfiguration.html) +and +[`advancedConfiguration.connectionTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.AdvancedBaseClientConfiguration.html) +still bound client-native work. + +### Native operation budgets + +DialCache's read deadline bounds its caller wait, not the complete lifetime of +the underlying client work. Configure finite client-native budgets for: + +- connection establishment; +- reconnection and retries; +- offline queueing; +- dispatch; and +- response time. + +For node-redis 4.7, `socket.connectTimeout`, `disableOfflineQueue`, and +`commandsQueueMaxLength` bound connection or queue behavior but do not impose a +strict response deadline after dispatch. Use client-native shutdown or +termination behavior that matches the application's resource and ambiguity +requirements. + +Redis writes and invalidations, asynchronous `cacheConfigProvider` work, and +custom `Serializer` methods still require their own finite budgets. Do not put +writes or invalidations behind a bare `Promise.race`: rejecting the outer +promise neither removes queued work nor proves that a dispatched mutation did +not execute. + +DialCache's [fallback deadline](coalescing.md#fallback-deadlines) covers only the +source loader. Prefer resource-native budgets and cooperative cancellation for +every injected operation. + +## Serialization + +The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. +It exchanges serialized values as `string | Buffer` and does not expose +client-specific commands or wire encodings. + +Distinct untracked and tracked read/write Lua sources, the invalidation source, +and wire constants are exported from `dialcache/redis-protocol`. Custom adapters +can throw these root-exported error classes: + +- `DialCacheRedisPayloadError`; +- `DialCacheRedisPayloadEncodingError`; and +- `DialCacheRedisProtocolError`. + +They distinguish malformed payloads, unsupported encodings, and invalid Lua +reply domains in logs. DialCache records bounded `cache_read`, +`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. + +### Binary frame + +Redis values use a compact binary frame: + +```text +byte 1 format version +bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) +byte 10 payload encoding (0 = UTF-8, 1 = raw binary) +bytes 11... serialized payload +``` + +Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is +authoritative, so expiry metadata is not duplicated in the frame. + +The payload comes from the cache operation's serializer or `JsonSerializer` by +default. Custom serializers can return `string` or `Buffer`. Strings are +stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. +Adapters restore the same representation before calling `serializer.load`. + +### Default JSON behavior + +DialCache uses native `JSON.stringify` and `JSON.parse` by default. There is no +runtime validation pass, so the default adds no traversal beyond JSON +serialization itself. A top-level `undefined` result is supported with an +internal sentinel. + +When `serializer.load` rejects a Redis payload, DialCache: + +1. records a `serialization_load` error; +2. counts the read as a remote miss; +3. runs the fallback; and +4. attempts to replace the rejected payload. + +A validating custom serializer can therefore treat an incompatible cached +value as a refreshable miss without adding a schema version to the cache key. + +`JsonSerializer` validates JSON syntax only. It cannot detect that a +structurally valid payload came from an incompatible application value schema. +Applications that retain one `useCase` across deployments must keep +default-JSON values backward compatible. + +For an incompatible change, either: + +- provide a serializer whose `load` method validates and rejects the old shape; + or +- change `useCase` to isolate the new cache entries. + +During a mixed deployment, mutually incompatible validating serializers can +repeatedly reject and replace each other's values. Correctness is preserved, +but expect additional fallback and Redis-write load until the rollout +converges. + +### Typed serializer requirement + +When a cached function or inline loader's resolved return type is statically +JSON-compatible, `serializer` is optional. This includes JSON primitives, +arrays, plain object or interface shapes, optional object fields, and a +top-level `undefined`. + +Types known not to survive the default round trip require a typed +`Serializer`: + +```ts +import { DialCache, type Serializer } from "dialcache"; + +const dialcache = new DialCache(); + +const dateSerializer: Serializer = { + dump: (value) => value.toISOString(), + load: (value) => + new Date(Buffer.isBuffer(value) ? value.toString("utf8") : value), +}; + +const getUpdatedAt = dialcache.cached( + (userId: string) => db.fetchUpdatedAt(userId), + { + keyType: "user_id", + useCase: "GetUpdatedAt", + cacheKey: (userId) => userId, + serializer: dateSerializer, + }, +); +``` + +The compile-time guard rejects known incompatible shapes such as: + +- `Date`, `Map`, and `Set`; +- `bigint`, symbols, and functions; +- Buffers and typed arrays; +- method-bearing class instances; +- required nested `undefined`; and +- `unknown` and `any`. + +The guard applies to every `cached()` declaration and `getOrLoad()` invocation +because active layers are selected at runtime. A global Redis serializer is not +parameterized by each returned type, so it cannot discharge this requirement. +Non-JSON operations must select a typed serializer. + +This guard is deliberately conservative rather than a proof of runtime data. +TypeScript cannot detect non-finite numbers, cyclic or shared references, +runtime getters, `toJSON` behavior, or data-only class instances that resemble +plain objects. Opaque, generic, or deeply recursive types may also require an +explicit serializer. + +Providing `Serializer`, including an explicitly typed +`JsonSerializer`, is a trusted caller assertion. DialCache does not perform +an additional serialize-and-deserialize cycle to validate it. From 183395bca67916a033f59ddc7bc1808ccbaf7416 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:26:48 -0700 Subject: [PATCH 2/7] docs: improve onboarding and reference coverage --- README.md | 98 +++++++++++++++++++------------------------ docs/configuration.md | 68 ++++++++++++++++++++++++++++++ docs/maintainers.md | 37 ++++++++++++++++ docs/observability.md | 22 +++++++++- docs/redis.md | 44 ++++++++++++++----- 5 files changed, 201 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index e676737..43391e3 100644 --- a/README.md +++ b/README.md @@ -23,35 +23,14 @@ invalidation policy, and resource budgets. ## Safety comes from explicit controls -- **Off by default.** Outside `dialcache.enable(...)`, both cached wrappers and - inline loaders are true pass-throughs: DialCache does not build a key, - resolve config, access a cache, or coalesce the call. Inside an enabled - scope, a layer still needs an effective policy before it participates. -- **Gradual and reversible rollout.** Configure TTL and ramp independently for - the process-local and remote layers. A ramp of `0` is off, `100` is fully on, - and `DialCacheKeyConfig.disabled()` is the all-layer policy kill switch. -- **Fail-open cache path.** Key, config, cache-read, and serialization-load - failures fall through to the source loader. Cache-write, - serialization-dump, logging, and metrics failures do not replace an otherwise - usable fallback result. Explicit remote invalidation failures are rethrown so - callers never assume a mutation was made safe when it was not. -- **Bounded defaults.** The process-local cache has a 10,000-entry default cap, - active remote reads have a 50-millisecond default deadline, and enabled - fallback executions have a 60-second default deadline. The read deadline - bounds DialCache's wait, not necessarily the underlying Redis command; - applications still need resource-native budgets for client work, config - providers, serializers, and source I/O. - -Use DialCache when you want to: - -- add caching to database or service reads without scattering cache get/set - plumbing across call sites; -- begin with one layer or a small deterministic key cohort, observe it, and - expand or reverse the rollout per use case; -- combine request-local, process-local, and shared caching behind one key and - policy contract; or -- coalesce hot-key misses, invalidate related Redis entries, and emit bounded - cache metrics without rebuilding those mechanisms for every function. +- **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the + loader without building a key, resolving policy, accessing a cache, or + coalescing work. +- **Gradual and reversible.** Start either shared layer at `0`, expand it by a + stable subset of keys, and turn every cache layer off through runtime policy. +- **Fail-open cache path.** Cache-plumbing failures fall through to the loader + instead of replacing a usable result. Explicit invalidation failures still + surface to the caller. ## Contents @@ -66,7 +45,7 @@ Use DialCache when you want to: ## Install ```bash -pnpm add dialcache +npm install dialcache ``` DialCache requires Node.js 22.0.0 or newer. Production deployments should use a @@ -80,6 +59,9 @@ clients application-owned: ## Quick start +Create one long-lived `DialCache` instance for each cache and coalescing domain, +typically once per service process: + ```ts import { DialCache, DialCacheKeyConfig } from "dialcache"; @@ -98,14 +80,18 @@ const getUser = dialcache.cached( // Outside enable(), this is a true pass-through to db.fetchUser: await getUser("123"); -// Inside enable(), the active cache layers participate: -const user = await dialcache.enable(() => getUser("123")); +// Inside enable(), the first call loads and the second reuses the cached value: +const user = await dialcache.enable(async () => { + await getUser("123"); // db.fetchUser, then populate process-local cache + return await getUser("123"); // process-local hit +}); ``` `cached(fn, options)` preserves the function's parameters and returns a Promise-based wrapper. `DialCacheKeyConfig.enabled(60)` gives the process-local -and remote layers a 60-second baseline TTL; the remote layer participates only -when a Redis or Valkey client is configured. +and remote layers a 60-second baseline TTL. It does not enable request-local +memoization, and the remote layer participates only when a Redis or Valkey +client is configured. For a one-shot calculation that should remain inline, [`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) @@ -133,11 +119,26 @@ previous state when their callbacks settle. cached before a mutation. Use the appropriate invalidation or TTL policy before serving later reads of mutable data. +### From local trial to production + +A typical adoption path is: + +1. start with the process-local cache shown above; +2. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) + when values should be shared across processes or hosts; +3. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) + before increasing production exposure; and +4. connect an application-owned runtime configuration source, then ramp a + stable subset of keys as described next. + ## Dial caching up or down Every cache operation can declare a stable `defaultConfig`. An optional `cacheConfigProvider` returns a sparse runtime overlay for the current key, so -policy can change independently of the loader: +policy can change independently of the loader. + +The example below focuses on runtime policy. Remote ramp settings take effect +only when a Redis or Valkey client is configured. ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; @@ -186,41 +187,28 @@ runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); In production, the provider can read from an application-owned dynamic config client instead of an in-memory map. DialCache resolves one policy snapshot per -enabled invocation. Keep the provider cheap and give any asynchronous work its -own finite budget. +enabled invocation. -For the process-local and remote layers: - -- a missing effective TTL disables that layer by policy; -- a configured TTL with no ramp defaults to `100`; -- `0` disables the layer; -- `100` enables the layer for every key; and -- an intermediate ramp uses DialCache's deterministic key-and-layer - assignment. +A shared layer needs an effective TTL. With a TTL but no ramp, it defaults to +`100`; a ramp of `0` disables it, `100` selects every key, and an intermediate +value selects a stable key cohort for that layer. Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% of calls. Increasing or decreasing a ramp preserves membership for keys that remain inside the threshold, and local and remote cohorts are layer-specific. DialCache keeps the assignment stable across releases. -If an application needs an externally coordinated cohort, its -`cacheConfigProvider` can return a per-key ramp override of `0` or `100`. Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing entries rather than deleting them; a later ramp-up can reuse entries that remain valid. Request-local caching is controlled separately by the `requestLocal` boolean. `DialCacheKeyConfig.disabled()` sets it to `false` and ramps both shared layers -to `0`. Provider errors do not silently activate the baseline: the invocation -records a config error and runs the source loader uncached. - -Remote-read waiting is runtime-controlled too. An overlay -`remoteReadTimeoutMs` takes precedence over the operation's `defaultConfig`, -then the instance's `redis.readTimeoutMs`, then the 50-millisecond core default. -Remote reads always have a finite positive deadline. +to `0`. See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) -for sparse-overlay precedence, validation, and layer behavior. +for sparse-overlay precedence, provider failure behavior, externally +coordinated cohorts, remote-read deadlines, and layer validation. ## How the read path works diff --git a/docs/configuration.md b/docs/configuration.md index fbaf568..5b42998 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -74,6 +74,39 @@ value meaning and serialization. Prefer `cached()` for reusable loaders and `getOrLoad()` for calculations intentionally local to one call site. +## Enable and disable scopes + +DialCache performs cache work only inside an enabled asynchronous scope. Create +each `DialCache` instance once and reuse it for the lifetime of its cache and +coalescing domain, typically one service process: + +| API | Behavior | +| --- | --- | +| `enable(fn)` | Enables caching for `fn` and the asynchronous work it awaits. The outermost call owns any request-local state. | +| `disable(fn)` | Temporarily restores pass-through behavior, commonly around nested mutation work. It does not evict existing values. | +| `isEnabled()` | Reports whether the current asynchronous call chain is inside a live enabled scope. | +| `withEnabled(fn)` | Exact alias for `enable(fn)`. | +| `withDisabled(fn)` | Exact alias for `disable(fn)`. | + +All five methods are instance-scoped. `enable()` and `disable()` always return a +`Promise`, including when their callback returns synchronously. Nested scopes +restore the previous state when their callbacks settle, and a nested +`enable()` inside `disable()` can opt a smaller read region back in. + +Enabled state follows Node's `AsyncLocalStorage`; it is not a process-global +flag. Once the outermost `enable()` callback settles, detached asynchronous work +that inherited the old context becomes pass-through and cannot repopulate its +closed request-local state. + +The root-exported `DialCacheContext` exposes the lower-level +`enable()`, `disable()`, and `isEnabled()` context primitive. It does not attach +itself to a `DialCache` instance or perform cache work. Most applications should +use the methods on `DialCache`. + +Keep mutation work outside the enabled boundary or inside `disable()`. Because +disabling does not evict existing values, mutable data still needs an +appropriate TTL or [targeted invalidation](invalidation.md) policy. + ## Keys, ids, and extra dimensions For `cached()`, the required `cacheKey` selector receives the wrapped @@ -337,6 +370,41 @@ Applications that need an externally coordinated cohort can use Ramping down bypasses affected entries; it does not evict them, so a later ramp-up can reuse entries that remain valid. +### Provider key input + +`cacheConfigProvider` receives the fully constructed, read-only `DialCacheKey` +for the invocation: + +| Field | Meaning | +| --- | --- | +| `namespace` | Logical application or environment namespace. | +| `keyType` and `id` | Primary identity. The selected id has already been converted to a string. | +| `args` | Secondary dimensions as normalized, name-sorted string pairs; entries whose value was `undefined` are omitted. | +| `useCase` | Stable operation name used in cache identity and metrics. | +| `prefix` | Encoded identity prefix, including a Redis Cluster hash tag when invalidation tracking is enabled. | +| `urn` | Complete encoded cache identity, including arguments and `useCase`. | +| `defaultConfig` | The operation's snapshotted baseline policy, or `null`. | +| `serializer` | The operation-specific serializer, or `null`. | +| `trackForInvalidation` | Whether the operation uses remote watermark tracking. | + +Use the identity fields to select policy; do not derive policy names or metric +dimensions from unbounded user input. The provider result remains a sparse +overlay and must not mutate the key. + +Most applications do not construct keys directly. Custom integrations can use +the root exports: + +- `new DialCacheKey(init)` to build the same public key shape; +- `normalizeArgs(record)` to omit `undefined`, stringify scalar values, and + sort argument names; +- `invalidationPrefix(namespace, keyType, id)` to build the encoded tracked + identity; and +- `redisClusterHashTag(value)` to wrap a validated value in a Redis Cluster hash + tag. + +The namespace and hash-tag components reject `{` and `}` as described under +[Identity rules](#identity-rules). + ## Request-local cache Set `requestLocal: true` to memoize resolved values for the lifetime of the diff --git a/docs/maintainers.md b/docs/maintainers.md index 4da5075..7db9422 100644 --- a/docs/maintainers.md +++ b/docs/maintainers.md @@ -2,6 +2,43 @@ [Back to the README](../README.md) +## Validation + +Use the repository's pinned pnpm version through Corepack: + +```bash +corepack pnpm install --frozen-lockfile +corepack pnpm check +corepack pnpm test:integration +``` + +`pnpm check` runs strict typechecking, the unit suite with coverage, +bundles/declarations, and packed ESM/CJS consumer tests. The integration suite +uses Testcontainers and requires a working Docker-compatible container runtime +for Redis, Valkey, and Redis Cluster. + +CI runs development and integration checks on Node.js 24, then switches to the +declared minimum Node.js 22.0.0 to test the packed package. Keep the consumer +floor separate from the development runtime so a new dependency or emitted +syntax cannot silently raise the published requirement. + +Before changing a compatibility-sensitive surface, identify and extend the +corresponding packed, unit, and integration assertions: + +- package root and explicit adapter/protocol entry points; +- full cache-key identity, encoding, namespace behavior, and Redis Cluster hash + tags; +- deterministic partial-ramp assignment, which must not reshuffle cohorts + across releases; +- the binary Redis frame, Lua arguments and reply domains, tracked + read/write/invalidation semantics, and mixed-version serializer behavior; and +- bounded metrics names, labels, reasons, error categories, scopes, and units. + +When changing user-facing examples, parse TypeScript fences, validate local +files and anchors, verify that README repository links are absolute for npm +rendering, and inspect the packed README. The package ships `README.md` but not +`docs/`. + ## Cache-path benchmark From a repository checkout, install dependencies and run: diff --git a/docs/observability.md b/docs/observability.md index ac08f1c..599a530 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -15,7 +15,7 @@ bounded labels. Install `prom-client` separately: ```bash -pnpm add prom-client@^15.1.3 +npm install prom-client@^15.1.3 ``` Create the registry your application owns, then pass an explicit adapter to @@ -105,7 +105,7 @@ in-flight state. Install `hot-shots` separately: ```bash -pnpm add hot-shots@^17.0.0 +npm install hot-shots@^17.0.0 ``` Create the DogStatsD client your application owns, then pass it to the Datadog @@ -253,6 +253,24 @@ and application fallback failures. Implement `DialCacheMetricsAdapter` and pass it through `new DialCache({ metrics })` for another telemetry backend. +| Hook | Required | Value | +| --- | --- | --- | +| `request(labels)` | yes | One active cache-layer lookup. | +| `miss(labels)` | yes | One cache miss. | +| `disabled(labels)` | yes | One skipped layer or no-layer invocation with a bounded `reason`. | +| `error(labels)` | yes | One bounded failure site with `inFallback`. | +| `invalidation(labels)` | yes | One explicit remote invalidation call. | +| `coalesced(labels)` | no | One follower that joined request-local or process-scoped work. | +| `observeGet(labels, seconds)` | yes | Cache-read duration in seconds. | +| `observeFallback(labels, seconds)` | yes | Fallback duration in seconds. | +| `observeSerialization(labels, seconds)` | yes | Serializer dump/load duration in seconds. | +| `observeSize(labels, bytes)` | yes | Serialized remote payload size in bytes. | + +The root package exports `DialCacheMetricsAdapter` and every associated label, +reason, error-kind, layer, and scope type. All hooks are synchronous; adapters +that buffer or transmit asynchronously own that later lifecycle. Keep label +values bounded and preserve the seconds and bytes units shown above. + Every backend-neutral label object exposes the logical namespace as camel-case `cacheNamespace`. Map it to the backend's `cache_namespace` label or tag. This field is present even when no key or cache layer was reached. diff --git a/docs/redis.md b/docs/redis.md index bd3f81d..36c01ef 100644 --- a/docs/redis.md +++ b/docs/redis.md @@ -13,10 +13,10 @@ Choose one supported integration: ```bash # node-redis -pnpm add redis@~4.7.1 +npm install redis@~4.7.1 # or Valkey GLIDE -pnpm add @valkey/valkey-glide +npm install @valkey/valkey-glide ``` ## node-redis @@ -189,9 +189,9 @@ The `fallbackTimeoutMs` timer is separate and starts only if and when the source loader begins. The remote-read timer covers neither config resolution, serializer loading, the fallback, Redis writes, nor invalidation. -### Custom-client read context +### Custom-client contract -The semantic boundary exposes an optional second argument: +Custom adapters implement the complete client-agnostic semantic boundary: ```ts interface RedisReadContext { @@ -204,13 +204,26 @@ interface DialCacheRedisClient { request: RedisReadRequest, context?: RedisReadContext, ): Awaitable; + write(request: RedisWriteRequest): Awaitable; + invalidate(request: RedisInvalidationRequest): Awaitable; } ``` -The optional argument keeps existing one-argument custom clients structurally -compatible. Adapters should use the signal for cooperative cancellation where -their client supports it, but the core deadline remains authoritative when -they do not. +| Method | Required semantics | +| --- | --- | +| `read` | Return the serialized `string` or `Buffer`, or `null` for a miss. A tracked request includes `watermarkKey`; compare the value timestamp and watermark atomically. | +| `write` | Apply `cacheTtlMs` and record server time atomically. A tracked request includes `watermarkKey`; return `false` when the watermark rejects publication and `true` when the value was written. | +| `invalidate` | Advance `watermarkKey` monotonically to at least server time plus `futureBufferMs`, while preserving the required derived lifetime. Reject on failure. | + +`write()` returning `false` is a safe publication refusal, not an adapter error. +DialCache still returns the fallback value but skips the corresponding +process-local population. A thrown cache-write error fails open; a thrown +explicit invalidation error is rethrown to the caller. + +The optional `RedisReadContext` keeps existing one-argument readers +structurally compatible. Adapters should use its signal for cooperative +cancellation where their client supports it, but the core deadline remains +authoritative when they do not. The bundled node-redis adapter forwards the signal in per-command options. This can remove queued work where supported, but aborting after dispatch cannot @@ -256,9 +269,18 @@ The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not expose client-specific commands or wire encodings. -Distinct untracked and tracked read/write Lua sources, the invalidation source, -and wire constants are exported from `dialcache/redis-protocol`. Custom adapters -can throw these root-exported error classes: +The `dialcache/redis-protocol` entry point exports the exact bundled protocol +building blocks: + +- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; +- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; +- `INVALIDATE_CACHE_SCRIPT`; and +- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and + `REDIS_ENCODING_BINARY`. + +The scripts implement the atomic read, publication, invalidation, server-time, +and derived-watermark-lifetime behavior required above. Custom adapters can +throw these root-exported error classes: - `DialCacheRedisPayloadError`; - `DialCacheRedisPayloadEncodingError`; and From 0a833b2241c05d9784ac92ddcaa70832f17ef961 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:36:02 -0700 Subject: [PATCH 3/7] docs: refine rollout safety guidance --- README.md | 51 +++++++++++--------- docs/coalescing.md | 21 ++++---- docs/configuration.md | 4 +- docs/invalidation.md | 5 +- docs/observability.md | 6 ++- docs/redis.md | 110 ++++++++++++++++++++++-------------------- 6 files changed, 107 insertions(+), 90 deletions(-) diff --git a/README.md b/README.md index 43391e3..ce8e5de 100644 --- a/README.md +++ b/README.md @@ -26,8 +26,9 @@ invalidation policy, and resource budgets. - **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the loader without building a key, resolving policy, accessing a cache, or coalescing work. -- **Gradual and reversible.** Start either shared layer at `0`, expand it by a - stable subset of keys, and turn every cache layer off through runtime policy. +- **Gradual and reversible.** Start the process-local or remote layer at `0`, + expand it by a stable subset of keys, and turn every cache layer off through + runtime policy. - **Fail-open cache path.** Cache-plumbing failures fall through to the loader instead of replacing a usable result. Explicit invalidation failures still surface to the caller. @@ -63,7 +64,7 @@ Create one long-lived `DialCache` instance for each cache and coalescing domain, typically once per service process: ```ts -import { DialCache, DialCacheKeyConfig } from "dialcache"; +import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; const dialcache = new DialCache(); @@ -73,7 +74,9 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + }), }, ); @@ -88,10 +91,9 @@ const user = await dialcache.enable(async () => { ``` `cached(fn, options)` preserves the function's parameters and returns a -Promise-based wrapper. `DialCacheKeyConfig.enabled(60)` gives the process-local -and remote layers a 60-second baseline TTL. It does not enable request-local -memoization, and the remote layer participates only when a Redis or Valkey -client is configured. +Promise-based wrapper. The configuration above enables only the process-local +layer with a 60-second TTL; its omitted ramp defaults to `100`. Request-local +memoization and the remote layer remain off. For a one-shot calculation that should remain inline, [`getOrLoad()`](https://github.com/lan17/DialCache/blob/main/docs/configuration.md#one-shot-inline-loaders) @@ -124,12 +126,12 @@ serving later reads of mutable data. A typical adoption path is: 1. start with the process-local cache shown above; -2. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) - when values should be shared across processes or hosts; -3. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) +2. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) before increasing production exposure; and -4. connect an application-owned runtime configuration source, then ramp a - stable subset of keys as described next. +3. connect an application-owned runtime configuration source with the remote + ramp at `0`; then +4. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) + and ramp a stable subset of keys as described next. ## Dial caching up or down @@ -170,7 +172,7 @@ runtimePolicies.set( }), ); -// Later, ramp both shared layers to 100%. +// Later, ramp the process-local and remote layers to 100%. runtimePolicies.set( "GetUser", new DialCacheKeyConfig({ @@ -189,9 +191,9 @@ In production, the provider can read from an application-owned dynamic config client instead of an in-memory map. DialCache resolves one policy snapshot per enabled invocation. -A shared layer needs an effective TTL. With a TTL but no ramp, it defaults to -`100`; a ramp of `0` disables it, `100` selects every key, and an intermediate -value selects a stable key cohort for that layer. +The process-local and remote layers each need an effective TTL. With a TTL but +no ramp, a layer defaults to `100`; a ramp of `0` disables it, `100` selects +every key, and an intermediate value selects a stable key cohort. Ramps select key cohorts, not requests or load, so `10` does not guarantee 10% of calls. Increasing or decreasing a ramp preserves membership for keys that @@ -203,8 +205,8 @@ entries rather than deleting them; a later ramp-up can reuse entries that remain valid. Request-local caching is controlled separately by the `requestLocal` boolean. -`DialCacheKeyConfig.disabled()` sets it to `false` and ramps both shared layers -to `0`. +`DialCacheKeyConfig.disabled()` sets it to `false` and ramps both the +process-local and remote layers to `0`. See [Configuration and cache layers](https://github.com/lan17/DialCache/blob/main/docs/configuration.md) for sparse-overlay precedence, provider failure behavior, externally @@ -226,11 +228,11 @@ open. `enable()` scope. - A process-local hit returns from the `DialCache` instance's bounded LRU. - A process-local miss can read Redis and populate the process-local cache. -- A remote miss runs the fallback and attempts to populate active shared +- A remote miss runs the fallback and attempts to populate the active cache layers. - A remote read failure or timeout runs the fallback without a second Redis - operation. An untracked result may still populate process-local cache; a - tracked result does not, because watermark safety was not established. + operation. Tracked invalidation adds a stricter + [publication rule](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md#read-and-write-behavior). - Same-key concurrent work is coalesced at the lifetime of the first active layer. @@ -324,8 +326,9 @@ before enabling it in production. Concurrent callers with the same cache key share active work within the first active cache scope: one outer request for request-local caching, or one -`DialCache` instance for the shared layers. This mitigates hot-key stampedes -inside that scope; it is not cross-process coordination. +`DialCache` instance when process-local or remote caching is active. This +mitigates hot-key stampedes inside that scope; it is not cross-process +coordination. Same-key followers share the leader's remaining remote-read budget. The fallback deadline starts separately only if and when the source loader begins. diff --git a/docs/coalescing.md b/docs/coalescing.md index 4ce4c34..ca7ffaa 100644 --- a/docs/coalescing.md +++ b/docs/coalescing.md @@ -7,9 +7,10 @@ cache layer. It applies a finite deadline to each active remote read and a separate default deadline once an initially enabled invocation begins its fallback loader. -These mechanisms reduce duplicate source work and give active flights eventual -cleanup. They do not replace cross-process coordination, source-native -cancellation, application admission control, or backpressure. +These mechanisms reduce duplicate source work. Their deadlines help flights +settle, but eventual cleanup still requires finite application-owned budgets +for every injected operation. They do not replace cross-process coordination, +source-native cancellation, admission control, or backpressure. ## Request coalescing @@ -26,7 +27,8 @@ different outer request has a different request-local flight registry. ### Process scope When process-local or remote caching is active, same-key callers share work -within one `DialCache` instance before the first active shared layer. +within one `DialCache` instance before the first active process-local or remote +layer. This is reported as `scope="process"`, but it is instance-scoped: @@ -36,7 +38,7 @@ This is reported as `scope="process"`, but it is instance-scoped: ```ts await dialcache.enable(async () => { - // Same cold key and active shared layer: + // Same cold key and active process-local or remote layer: // one fallback execution, one shared result. const [first, second] = await Promise.all([ getUser("456"), @@ -202,10 +204,11 @@ requested. There is no library-wide flight cap or age-based replacement. A registry cap would bound only DialCache metadata. Overflow or eviction could -still create unbounded source work and unsafe duplicate publication. Finite -operation deadlines provide eventual cleanup; application admission control -and backpressure remain responsible for bounding simultaneous distinct-key -work. +still create unbounded source work and unsafe duplicate publication. +DialCache's remote-read and fallback deadlines cover only those phases; +provider, serializer, and Redis-write settlement remains application-owned. +Admission control and backpressure remain responsible for bounding +simultaneous distinct-key work. Monitor: diff --git a/docs/configuration.md b/docs/configuration.md index 5b42998..20b21d1 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -235,8 +235,8 @@ runtime field -> defaultConfig field -> DialCache disabled baseline ``` The disabled baseline sets `requestLocal` to `false` and leaves the -process-local and remote TTLs unset. A shared layer with no effective TTL is -disabled by policy. A shared layer with an effective TTL but no effective ramp +process-local and remote TTLs unset. Either layer is disabled by policy when it +has no effective TTL. With an effective TTL but no effective ramp, that layer defaults to a 100% ramp. The remote-read deadline has two additional fallbacks: diff --git a/docs/invalidation.md b/docs/invalidation.md index b853ec5..1652293 100644 --- a/docs/invalidation.md +++ b/docs/invalidation.md @@ -161,8 +161,9 @@ watermark TTL. Invalidation ensures the TTL covers both the requested future buffer and any still-future existing watermark, plus one minute. It also preserves a longer or -persistent TTL. There is no fixed or configurable retention floor, and reads do -not extend watermark lifetime. +persistent TTL. The one-minute safety margin is fixed; there is no separate +configurable or global retention floor, and reads do not extend watermark +lifetime. ## Choosing `futureBufferMs` diff --git a/docs/observability.md b/docs/observability.md index 599a530..ca361d9 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -135,8 +135,10 @@ const dialcache = new DialCache({ }), }); -// Drain outstanding cache operations before application shutdown. -dogStatsD.close(); +function shutdown(): void { + // Drain outstanding cache operations before application shutdown. + dogStatsD.close(); +} ``` `hot-shots` is the supported and tested client, but the adapter depends only on diff --git a/docs/redis.md b/docs/redis.md index 36c01ef..8f0b52e 100644 --- a/docs/redis.md +++ b/docs/redis.md @@ -4,8 +4,8 @@ DialCache's remote TTL layer supports standalone Redis, standalone Valkey, and Redis Cluster. The application creates, connects, configures, drains, and closes -the underlying client. DialCache borrows a semantic `DialCacheRedisClient` and -does not own the connection lifecycle. +the underlying client. DialCache borrows a client-independent +`DialCacheRedisClient` adapter and does not own the connection lifecycle. ## Install a client @@ -22,7 +22,7 @@ npm install @valkey/valkey-glide ## node-redis Register DialCache's native scripts when creating the client, connect it, and -pass the semantic adapter to `DialCache`: +pass the DialCache-compatible adapter to `DialCache`: ```ts import { createClient } from "redis"; @@ -144,7 +144,7 @@ client-side invocation has settled. ## Remote-read deadlines and async liveness -Every active semantic remote-read leader has a finite monotonic deadline. +Every active remote-read leader has a finite monotonic deadline. DialCache uses this precedence for `cached()` and `getOrLoad()`: ```text @@ -191,7 +191,8 @@ serializer loading, the fallback, Redis writes, nor invalidation. ### Custom-client contract -Custom adapters implement the complete client-agnostic semantic boundary: +Custom adapters implement the complete client-independent read, write, and +invalidate contract: ```ts interface RedisReadContext { @@ -211,9 +212,9 @@ interface DialCacheRedisClient { | Method | Required semantics | | --- | --- | -| `read` | Return the serialized `string` or `Buffer`, or `null` for a miss. A tracked request includes `watermarkKey`; compare the value timestamp and watermark atomically. | -| `write` | Apply `cacheTtlMs` and record server time atomically. A tracked request includes `watermarkKey`; return `false` when the watermark rejects publication and `true` when the value was written. | -| `invalidate` | Advance `watermarkKey` monotonically to at least server time plus `futureBufferMs`, while preserving the required derived lifetime. Reject on failure. | +| `read` | Return the serialized `string` or `Buffer`, or `null` for a miss. For a tracked request, compare the value timestamp and watermark atomically; a missing watermark or a value at or behind it is a miss. | +| `write` | Apply `cacheTtlMs` and record server time atomically. For a tracked request, create a missing baseline, retain it for at least the value TTL plus one minute without shortening a longer or persistent lifetime, and return `false` when it rejects publication. Return `true` only when the value was written. | +| `invalidate` | Advance `watermarkKey` monotonically to at least server time plus `futureBufferMs`. Retain it long enough to cover that buffer and any still-future existing watermark, plus one minute, without shortening a longer or persistent lifetime. Reject on failure. | `write()` returning `false` is a safe publication refusal, not an adapter error. DialCache still returns the fallback value but skips the corresponding @@ -265,49 +266,10 @@ every injected operation. ## Serialization -The core Redis boundary is the client-agnostic `DialCacheRedisClient` interface. -It exchanges serialized values as `string | Buffer` and does not expose -client-specific commands or wire encodings. - -The `dialcache/redis-protocol` entry point exports the exact bundled protocol -building blocks: - -- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; -- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; -- `INVALIDATE_CACHE_SCRIPT`; and -- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and - `REDIS_ENCODING_BINARY`. - -The scripts implement the atomic read, publication, invalidation, server-time, -and derived-watermark-lifetime behavior required above. Custom adapters can -throw these root-exported error classes: - -- `DialCacheRedisPayloadError`; -- `DialCacheRedisPayloadEncodingError`; and -- `DialCacheRedisProtocolError`. - -They distinguish malformed payloads, unsupported encodings, and invalid Lua -reply domains in logs. DialCache records bounded `cache_read`, -`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. - -### Binary frame - -Redis values use a compact binary frame: - -```text -byte 1 format version -bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) -byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload -``` - -Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is -authoritative, so expiry metadata is not duplicated in the frame. - -The payload comes from the cache operation's serializer or `JsonSerializer` by -default. Custom serializers can return `string` or `Buffer`. Strings are -stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. -Adapters restore the same representation before calling `serializer.load`. +DialCache uses `JsonSerializer` by default. A cache operation can select a +typed serializer, and `redis.serializer` supplies the instance default when an +operation does not select one. Serializers run only for remote reads and +writes; request-local and process-local values remain native references. ### Default JSON behavior @@ -397,3 +359,49 @@ explicit serializer. Providing `Serializer`, including an explicitly typed `JsonSerializer`, is a trusted caller assertion. DialCache does not perform an additional serialize-and-deserialize cycle to validate it. + +### Advanced wire protocol + +The core Redis boundary is the client-independent `DialCacheRedisClient` +interface. It exchanges serialized values as `string | Buffer` and does not +expose client-specific commands or wire encodings. + +The `dialcache/redis-protocol` entry point exports the exact bundled protocol +building blocks: + +- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; +- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; +- `INVALIDATE_CACHE_SCRIPT`; and +- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and + `REDIS_ENCODING_BINARY`. + +The scripts implement the atomic read, publication, invalidation, server-time, +and derived-watermark-lifetime behavior required above. Custom adapters can +throw these root-exported error classes: + +- `DialCacheRedisPayloadError`; +- `DialCacheRedisPayloadEncodingError`; and +- `DialCacheRedisProtocolError`. + +They distinguish malformed payloads, unsupported encodings, and invalid Lua +reply domains in logs. DialCache records bounded `cache_read`, +`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. + +#### Binary frame + +Redis values use a compact binary frame: + +```text +byte 1 format version +bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) +byte 10 payload encoding (0 = UTF-8, 1 = raw binary) +bytes 11... serialized payload +``` + +Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is +authoritative, so expiry metadata is not duplicated in the frame. + +The payload comes from the cache operation's serializer or `JsonSerializer` by +default. Custom serializers can return `string` or `Buffer`. Strings are +stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. +Adapters restore the same representation before calling `serializer.load`. From 6502c7c82ac3d9c268dbc4a33eaf20212d9e2502 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:43:07 -0700 Subject: [PATCH 4/7] docs: make rollout examples fail safe --- README.md | 23 ++++++++++++++++++----- docs/configuration.md | 7 ++++--- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index ce8e5de..c83595f 100644 --- a/README.md +++ b/README.md @@ -60,8 +60,9 @@ clients application-owned: ## Quick start -Create one long-lived `DialCache` instance for each cache and coalescing domain, -typically once per service process: +Most services create one long-lived `DialCache` instance and reuse it across +the process. It owns one process-local LRU and one process-coalescing scope; +create separate instances only when those resources should be isolated: ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; @@ -128,8 +129,8 @@ A typical adoption path is: 1. start with the process-local cache shown above; 2. add [Prometheus or Datadog](https://github.com/lan17/DialCache/blob/main/docs/observability.md) before increasing production exposure; and -3. connect an application-owned runtime configuration source with the remote - ramp at `0`; then +3. extend the policy with a remote TTL and a remote ramp of `0`, using an + application-owned runtime configuration source; then 4. add [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) and ramp a stable subset of keys as described next. @@ -157,7 +158,16 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, - defaultConfig: DialCacheKeyConfig.enabled(60), + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { + [CacheLayer.LOCAL]: 60, + [CacheLayer.REMOTE]: 60, + }, + ramp: { + [CacheLayer.LOCAL]: 0, + [CacheLayer.REMOTE]: 0, + }, + }), }, ); @@ -187,6 +197,9 @@ runtimePolicies.set( runtimePolicies.set("GetUser", DialCacheKeyConfig.disabled()); ``` +The zero-ramp baseline is the safety net: if the provider has no matching +entry, both layers remain off. + In production, the provider can read from an application-owned dynamic config client instead of an in-memory map. DialCache resolves one policy snapshot per enabled invocation. diff --git a/docs/configuration.md b/docs/configuration.md index 20b21d1..3ac1c44 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -76,9 +76,10 @@ intentionally local to one call site. ## Enable and disable scopes -DialCache performs cache work only inside an enabled asynchronous scope. Create -each `DialCache` instance once and reuse it for the lifetime of its cache and -coalescing domain, typically one service process: +DialCache performs cache work only inside an enabled asynchronous scope. Most +services create one instance and reuse it for the service process. Each +instance owns one process-local LRU and one process-coalescing registry; create +separate instances only to isolate those resources: | API | Behavior | | --- | --- | From e639a05d1857eda46a7e33395ac1d0193f060853 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:54:16 -0700 Subject: [PATCH 5/7] docs: clarify read-through cache positioning --- README.md | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index c83595f..10a109b 100644 --- a/README.md +++ b/README.md @@ -4,31 +4,37 @@ [![Codecov](https://codecov.io/gh/lan17/DialCache/branch/main/graph/badge.svg)](https://codecov.io/gh/lan17/DialCache) [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/lan17/DialCache/badge)](https://scorecard.dev/viewer/?uri=github.com/lan17/DialCache) -**Roll out backend caching like a feature—not a leap of faith.** +**Read-through caching with the controls production systems need.** -**DialCache is** a TypeScript library for caching database and service reads -inside Node.js backends. It routes reusable async functions and inline loaders -through one read-through path with request-local memoization, a bounded -in-process LRU, and optional Redis or Valkey caching. +DialCache is a TypeScript read-through caching library for asynchronous +database and service reads in Node.js. Wrap a reusable function with +`cached()` or keep a loader inline with `getOrLoad()`; when the active cache +layers miss, DialCache calls your loader and publishes the result to whichever +request-local, bounded process-local, and optional Redis or Valkey layers are +active. -The “dial” is per-use-case runtime control. Start with caching off, dial the -process-local and remote layers up for stable cohorts of keys, and dial them -back down without changing the loader. +Around that core path, DialCache provides patterns that high-scale services +otherwise have to build themselves: request coalescing, per-use-case runtime +policy, deterministic ramp-up and ramp-down, fail-open cache access, targeted +invalidation, serialization, deadlines, and backend-neutral metrics. -**DialCache is not** a frontend data cache, cache server, Redis or Valkey -client, or runtime configuration service. It supplies the cache path and -rollout controls; your application still decides what is safe to cache and -owns loader behavior, connections, runtime configuration, keys, TTLs, -invalidation policy, and resource budgets. +The “dial” is the runtime policy: start a use case at zero, expand local or +remote caching to stable key cohorts, and reverse the rollout without changing +the loader. + +DialCache is a backend application library—not a frontend data cache, cache +server, Redis or Valkey client, or configuration control plane. Your service +owns the loader, clients, dynamic configuration source, cache identity, TTLs, +invalidation windows, admission control, and resource budgets. ## Safety comes from explicit controls - **Off by default.** Outside `dialcache.enable(...)`, calls go straight to the loader without building a key, resolving policy, accessing a cache, or coalescing work. -- **Gradual and reversible.** Start the process-local or remote layer at `0`, - expand it by a stable subset of keys, and turn every cache layer off through - runtime policy. +- **Gradual and reversible.** Start process-local and remote ramps at `0`, + expand either to a stable key cohort, and turn every cache layer back off + through runtime policy. - **Fail-open cache path.** Cache-plumbing failures fall through to the loader instead of replacing a usable result. Explicit invalidation failures still surface to the caller. From b44c9de34fae1344b063a9e5a6d0e3b8fdfadfc1 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Sat, 25 Jul 2026 20:55:30 -0700 Subject: [PATCH 6/7] docs: make README opening easier to scan --- README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 10a109b..1b83fc6 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,13 @@ **Read-through caching with the controls production systems need.** -DialCache is a TypeScript read-through caching library for asynchronous -database and service reads in Node.js. Wrap a reusable function with -`cached()` or keep a loader inline with `getOrLoad()`; when the active cache -layers miss, DialCache calls your loader and publishes the result to whichever -request-local, bounded process-local, and optional Redis or Valkey layers are -active. +DialCache is a TypeScript read-through caching library for async database and +service reads in Node.js. + +Wrap a reusable function with `cached()` or keep a loader inline with +`getOrLoad()`; when the active cache layers miss, DialCache calls your loader +and publishes the result to whichever request-local, bounded process-local, +and optional Redis or Valkey layers are active. Around that core path, DialCache provides patterns that high-scale services otherwise have to build themselves: request coalescing, per-use-case runtime From e6ec8492ff8ceef38883bd0a4fe19687643b8524 Mon Sep 17 00:00:00 2001 From: Lev Neiman Date: Wed, 12 Aug 2026 14:16:30 -0700 Subject: [PATCH 7/7] docs: refresh guides for v0.19.0 --- AGENTS.md | 7 +- README.md | 70 +++++-- docs/coalescing.md | 117 ++++++++--- docs/configuration.md | 167 ++++++++++++---- docs/invalidation.md | 62 +++++- docs/maintainers.md | 72 ++++++- docs/observability.md | 114 +++++++++-- docs/redis.md | 407 ++++++++++++++++++++++++++++++-------- docs/shadow-validation.md | 160 +++++++++------ 9 files changed, 918 insertions(+), 258 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 061674a..7572450 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -44,7 +44,12 @@ test/ # Unit and Redis integration tests - `invalidateRemote()` requires a configured Redis client and rejects when the client is absent or the watermark mutation fails. - Tracked Redis values and invalidation watermarks share a Redis Cluster hash tag. -- Tracked reads run on primaries so replica lag cannot hide invalidation. +- Tracked reads require one authoritative value/watermark snapshot; cluster + adapters explicitly route them to primaries so replica lag cannot hide + invalidation. +- Redis payload compression is default-on for writes, while reads always + interpret the compression envelope so disabling new compression does not + strand existing entries. - A tracked write's placeholder frame (version byte 0) is unreadable on both read paths until the stamp script promotes it, and the stamp promotes only the placeholder carrying its own per-write nonce. diff --git a/README.md b/README.md index c7ac8b4..12e8006 100644 --- a/README.md +++ b/README.md @@ -17,8 +17,8 @@ and optional Redis or Valkey layers are active. Around that core path, DialCache provides patterns that high-scale services otherwise have to build themselves: request coalescing, per-use-case runtime policy, deterministic ramp-up and ramp-down, detached shadow validation, -fail-open cache access, targeted invalidation, serialization, deadlines, and -backend-neutral metrics. +fail-open cache access, targeted invalidation, transparent Redis compression, +deadlines, and backend-neutral metrics. The “dial” is the runtime policy: start a use case at zero, expand local or remote caching to stable key cohorts, and reverse the rollout without changing @@ -58,7 +58,8 @@ invalidation windows, admission control, and resource budgets. npm install dialcache ``` -DialCache requires Node.js 22.0.0 or newer. Production deployments should use a +DialCache requires Node.js `>=22.15.0 <23.0.0 || >=23.8.0`; Node.js 23.0 +through 23.7 lack the `node:zlib` zstd API. Production deployments should use a [currently supported LTS release](https://nodejs.org/en/about/previous-releases). Redis, Valkey, Prometheus, and Datadog integrations are optional and keep their @@ -144,7 +145,7 @@ the initial production rollout policy: 3. add a remote TTL and [Redis or Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) while the remote serving ramp remains `0`; -4. for tracked use cases, optionally +4. optionally [validate and fill Redis in shadow mode](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) without serving it; and 5. increase process-local, shadow, and remote cohorts independently while @@ -237,10 +238,12 @@ Ramping down, including with `DialCacheKeyConfig.disabled()`, bypasses existing entries rather than deleting them; a later ramp-up can reuse entries that remain valid. -Request-local caching is controlled separately by the `requestLocal` boolean. -`DialCacheKeyConfig.disabled()` sets it to `false`, sets `shadow.ramp` to `0` -and `shadow.logMismatches` to `false`, and ramps both the process-local and -remote layers to `0`. +Request-local caching and in-flight sharing are controlled separately. +`requestLocal` defaults to `false`; `coalesce` defaults to `true`. +`DialCacheKeyConfig.disabled()` turns request-local caching and shadow work off +and ramps both shared layers to `0`. It leaves `coalesce` unset, so a later +runtime ramp-up returns to default-on coalescing unless policy explicitly opts +out. A remote serving ramp of `0` alone does not override an inherited nonzero `shadow.ramp`; set both ramps to `0` to stop new invocation-driven Redis reads @@ -252,8 +255,8 @@ coordinated cohorts, remote-read deadlines, and layer validation. ## Validate Redis before serving it -For invalidation-tracked use cases, shadow mode can exercise Redis before Redis -is allowed to serve callers. On a selected tracked Redis hit, DialCache returns +Shadow mode can exercise tracked or untracked Redis entries before Redis is +allowed to serve callers. On a selected Redis hit, DialCache returns the cached value first, then compares a fresh decoding of the retained payload with a detached source read. @@ -265,7 +268,10 @@ sampled by `shadow.ramp`, bounded per instance by `shadowMaxInFlight`, and disabled unless the metrics adapter implements the shadow outcome hook. Shadow validation can add source and Redis work, remains best-effort during -shutdown, and requires tracked keys plus a valid remote TTL. +shutdown, and requires a valid remote TTL. Tracked keys retain watermark +fencing; untracked shadow fills use ordinary TTL-based last-writer-wins writes. +If upgrading from before `v0.15.0`, set `shadow.ramp` to `0` first when +untracked shadow reads and fills have not yet been approved. Confirmed mismatch warnings are separately opt-in through `shadow.logMismatches`. They can include logical cache keys and JSON-serialized @@ -294,14 +300,14 @@ open. - A process-local miss can read Redis and populate the process-local cache. - A remote miss runs the fallback and attempts to populate the active cache layers. -- Selected tracked keys can schedule detached shadow work after a Redis serving - hit or when the Redis serving ramp excludes the key. Shadow work never serves - the caller; it can validate a hit or fill a clean miss. +- Selected tracked or untracked keys can schedule detached shadow work after a + Redis serving hit or when the Redis serving ramp excludes the key. Shadow + work never serves the caller; it can validate a hit or fill a clean miss. - A caller-path remote read failure or timeout runs the fallback without a second caller-path Redis operation. Tracked invalidation adds a stricter [publication rule](https://github.com/lan17/DialCache/blob/main/docs/invalidation.md#read-and-write-behavior). - Same-key concurrent work is coalesced within the scope of the first active - layer. + layer unless the resolved policy sets `coalesce: false`. When all serving layers are disabled by policy, an initially enabled call remains uncached and uncoalesced even if selected shadow work runs @@ -365,6 +371,18 @@ scopes with bounded key cardinality. Process-local values count toward one instance-wide entry cap. Remote values use a serializer selected by the cache operation or the Redis configuration. +Redis payloads at least 4 KiB are compressed with zstd by default, but only +when the encoded value becomes smaller. Compression runs synchronously on the +Node.js event loop, so tune the threshold for your payload and latency profile +or set `redis.compression` to `false`. + +Reads still decode marked compressed entries after writes are disabled, which +means `compression: false` does not strand entries for readers that understand +the envelope. Older package versions are a separate mixed-deployment and +rollback concern. See +[Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md#compression) +for configuration, metrics, size limits, and custom-serializer compatibility. + Shadow validation uses detached Redis work but is not another serving `CacheLayer`. Its sampling, capacity, deduplication, deadline, and metrics contracts are independent of request coalescing and the remote serving ramp. @@ -410,6 +428,12 @@ coordination. Same-key followers share the leader's remaining remote-read budget. The fallback deadline starts separately only if and when the source loader begins. +Set `coalesce: false` only when concurrent calls with the same cache key must +not share caller-specific execution. Each caller then performs independent +layer reads, fallback work, deadlines, and writes; settled request-local values +can still serve later sequential calls. This also gives up stampede protection +for that use case. + Enabled fallbacks have a 60-second monotonic deadline by default. Timing out rejects the DialCache chain and prevents the late result from being published, but it does not cancel the underlying function. Give source operations their @@ -423,8 +447,8 @@ for exact sharing, deadline, cleanup, and admission-control contracts. Metrics are disabled unless a `DialCacheMetricsAdapter` is supplied. First-party adapters support caller-owned Prometheus registries and Datadog DogStatsD clients. Their fixed schemas report layer requests, misses, disabled reasons, -coalescing scopes, serialization work, shadow outcomes, and cache versus -fallback failures. +coalescing scopes, serialization and compression work, shadow outcomes, and +cache versus fallback failures. Keep application-owned namespaces, use-case names, and key types stable and low-cardinality. Optional confirmed-mismatch warnings are value-bearing logs, @@ -451,14 +475,20 @@ Before ramping a use case: - use a conservative `localMaxSize` and bounded request-local scopes; - treat cached values as immutable; - verify serializer compatibility across mixed application versions; +- benchmark synchronous compression for representative payloads and monitor + compression outcomes, prepared payload size, and ratio before changing its + threshold; +- set `coalesce: false` only when independent same-key execution is required + and the resulting source and Redis fan-out is acceptable; - before enabling shadow mode, confirm the loader is safe for an extra observational read, preserve immutable inputs and results, bound concurrency, and monitor added load and outcomes; - before enabling shadow mismatch logging, approve how logical keys and serialized values are redacted, transported, accessed, and retained; - plan shutdown around detached shadow work and application-owned dependencies; and -- for tracked invalidation, use synchronized Redis clocks, durable non-evictable - watermarks, and an application-sized nonzero buffer. +- for tracked invalidation, use authoritative primary reads, synchronized Redis + clocks, durable non-evictable watermarks, and an application-sized nonzero + buffer. ## Reference guides @@ -466,7 +496,7 @@ Before ramping a use case: runtime overlays, request-local and process-local behavior, and value ownership. - [Redis and Valkey](https://github.com/lan17/DialCache/blob/main/docs/redis.md) — node-redis and GLIDE setup, lifecycle, - liveness, binary protocol, and serialization. + liveness, native command protocol, serialization, and compression. - [Shadow validation and Redis bootstrap](https://github.com/lan17/DialCache/blob/main/docs/shadow-validation.md) — non-serving rollout, eligibility, comparison, clean-miss filling, capacity, deadlines, metrics, mismatch diagnostics, and lifecycle. diff --git a/docs/coalescing.md b/docs/coalescing.md index 5f6c133..882d81a 100644 --- a/docs/coalescing.md +++ b/docs/coalescing.md @@ -2,10 +2,10 @@ [Back to the README](../README.md) -DialCache shares same-key in-flight work within the lifetime of the first active -cache layer. It applies a finite deadline to each active remote read and a -separate default deadline once an initially enabled invocation begins its -fallback loader. +By default, DialCache shares same-key in-flight work within the lifetime of the +first active cache layer. A per-use-case policy can disable that sharing. Each +active remote read has a finite deadline, and a separate default deadline begins +when an initially enabled invocation starts its fallback loader. These mechanisms reduce duplicate source work. Their deadlines help flights settle, but eventual cleanup still requires finite application-owned budgets @@ -21,17 +21,18 @@ DialCache has two sharing scopes. ### Request-local scope -When request-local caching is active, callers with the same key in one outermost -`enable()` scope share in-flight work before the request-local lookup. +When request-local caching is active and coalescing is enabled, callers with the +same key in one outermost `enable()` scope share in-flight work before the +request-local lookup. The resolved value is memoized for later sequential calls in that scope. A different outer request has a different request-local flight registry. ### Process scope -When process-local or remote caching is active, same-key callers share work -within one `DialCache` instance before the first active process-local or remote -layer. +When process-local or remote caching is active and coalescing is enabled, +same-key callers share work within one `DialCache` instance before the first +active process-local or remote layer. This is reported as `scope="process"`, but it is instance-scoped: @@ -58,29 +59,85 @@ cache write; followers await that result. For a process-local-only miss, followers share the leader's fallback and local write. This mitigates a thundering herd on one hot key within the instance. +### Per-use-case opt-out + +`DialCacheKeyConfig.coalesce` is a sparse runtime boolean whose effective +default is `true`. Set it to `false` in a use case's `defaultConfig` or runtime +overlay to disable both request-local and process-scoped single-flight: + +```ts +import { CacheLayer, DialCacheKeyConfig } from "dialcache"; + +const getUser = dialcache.cached( + (userId: string) => db.fetchUser(userId), + { + keyType: "user_id", + useCase: "GetUserWithoutSingleFlight", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + coalesce: false, + }), + }, +); +``` + +Concurrent same-key callers then each perform: + +- their own active-layer reads with a full independent remote-read budget; +- their own fallback, error, and fallback deadline when a fallback is needed; +- their own cache writes after a miss. + +Request-local and process-local publication is last-writer-wins. Each Redis +write keeps its ordinary TTL-based or watermark-fenced semantics. A settled +request-local value can still serve a later sequential call in the same outer +scope; the policy disables in-flight sharing, not memoization or cache hits. + +Runtime overlays can explicitly change the field in either direction. Omission +inherits the baseline and ultimately defaults to `true`. +`DialCacheKeyConfig.disabled()` deliberately leaves `coalesce` unset: with every +serving layer off there is no flight to share, and a later runtime ramp-up +coalesces again unless it explicitly opts out. + +The public constructor and static `defaultConfig` validation require a boolean +when the field is present. A malformed runtime value fails config resolution for +the whole invocation: DialCache warns, records `config_resolution` and +`config_error`, and executes the fallback uncached without touching Redis. + +Use the opt-out when executions with the same value identity must not inherit a +leader's failure, cancellation behavior, or `FallbackTimeoutError`. It does not +make an incomplete cache key safe: if an input changes the returned value, put +it in the key or disable the affected cache layers. Disabling coalescing +reintroduces thundering-herd exposure, independent Redis load, and write races. + +No metric or state surface is added. An opted-out use case emits no +`coalesced` event, records request, miss, and latency observations once per +caller rather than once per flight, and does not register process state in +`getCoalescingState()`. + ## When calls do not coalesce -Coalescing applies only when at least one cache layer is active: +Coalescing applies only when at least one cache layer is active and the resolved +`coalesce` policy is not `false`: - calls that start outside `enable()` are true pass-through; - initially enabled calls with every layer disabled are uncached and - uncoalesced; and + uncoalesced; +- a use case with `coalesce: false` keeps each caller's cache path independent; - process-scoped work is never shared across `DialCache` instances. An initially enabled all-disabled call still receives the fallback deadline described below. -Because coalescing is keyed by the full constructed cache key, concurrent calls -with the same identity share the leader's execution. Every function argument -or captured value omitted from the selected or direct key must be safe to share -this way. +The full constructed cache key always defines cached-value identity. Include +locale, auth context, or any other input that can change the returned value, +regardless of the coalescing policy. -Include locale, auth context, cancellation behavior, or any other input in the -key when it can change: - -- the returned value; -- whether the underlying function should run independently; or -- whether two callers may safely share one result. +When coalescing is enabled, that same key also defines execution identity: +concurrent calls with the same key share the leader's execution. Include +cancellation behavior and other execution-only inputs when they must differ by +key, or use `coalesce: false` when their results remain safe to cache under the +same value identity but their in-flight work must stay independent. ### Shadow work does not enable caller coalescing @@ -93,9 +150,12 @@ shadow jobs are deduplicated by admitting one and reporting the others as `dropped`; callers do not join or await that job. A serving Redis hit reached through a process-scoped leader schedules at most -one shadow job for its coalesced followers. `shadowMaxInFlight` limits scheduled -or running shadow jobs across the instance, independently of request-local and -process-scoped flights. See +one shadow job for its coalesced followers. With `coalesce: false`, each caller +can attempt to schedule validation, but exact-key shadow deduplication admits at +most one concurrent job and reports the other attempts as `dropped`. + +`shadowMaxInFlight` limits scheduled or running shadow jobs across the +instance, independently of request-local and process-scoped flights. See [Shadow validation and Redis bootstrap](shadow-validation.md) for the full admission and lifecycle contract. @@ -142,6 +202,8 @@ The timer starts only when the fallback begins: - same-key followers share the request-local or process leader's remaining budget and receive its `FallbackTimeoutError`; +- callers with `coalesce: false` start independent fallback timers and receive + independent errors; - a remote read failure or timeout starts the fallback timer only when the source loader begins; - enabled pass-through invocations where every layer is disabled have @@ -178,7 +240,7 @@ shutdown requirements. Timing out: 1. rejects the DialCache chain; -2. clears its tracked flight normally; +2. clears its coalescing flight normally, when one exists; 3. ignores a later fallback resolution; and 4. prevents that invocation from proceeding to serializer, Redis, or local publication. @@ -196,7 +258,8 @@ Timeout failures retain the bounded metrics classification details without adding high-cardinality labels. A shared remote-read timeout emits one `cache_read_timeout` error for the -leader, not one per follower. +leader, not one per follower. With coalescing disabled, each caller owns its +read and can emit its own timeout error. ### Shadow deadlines are separate @@ -242,6 +305,8 @@ after that point. Request-local flights are deliberately excluded because their lifecycle is bounded by the outer `enable()` scope. Shadow jobs are also excluded; they use their own capacity registry and outcome metrics. +Use cases with `coalesce: false` never register process flights and therefore do +not appear in this state. `oldestLeaderAgeMs` uses a monotonic clock and is computed when the snapshot is requested. diff --git a/docs/configuration.md b/docs/configuration.md index 97b4eea..d6a5539 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -3,8 +3,9 @@ [Back to the README](../README.md) This guide covers reusable cached functions, one-shot inline loaders, cache -identity, runtime policy, request-local and process-local behavior, and -cached-value ownership. For the shared remote layer, see +identity, runtime policy, coalescing policy, request-local and process-local +behavior, Redis payload-compression configuration, and cached-value ownership. +For the complete shared remote-layer contract, see [Redis and Valkey](redis.md). ## Defining cache operations @@ -68,9 +69,13 @@ high-cardinality input because it is part of both cache identity and metrics labels. Put those values in `key` instead. Every captured value that can change the result belongs in the bare id or -`{ id, args }` key. Concurrent same-key calls may share one caller's in-flight -loader and cached value, so all call sites for that identity must also agree on -value meaning and serialization. +`{ id, args }` key. By default, concurrent same-key calls may share one +caller's in-flight loader and cached value, so all call sites for that identity +must also agree on value meaning and serialization. + +A use case can explicitly set `coalesce: false` when its callers must execute +independently, but that does not make an incomplete cache key safe for settled +cache hits. Shadow work can run the loader later, after the caller has continued. Snapshot mutable arguments or captured state before invoking the operation so that the @@ -189,9 +194,11 @@ characters for Redis Cluster hash tags. name or value. - **Non-key inputs still reach the loader.** A database handle can be a normal function parameter ignored by `cacheKey` or a value captured by a - `getOrLoad()` loader. Concurrent same-key misses share the leader's - execution. Do not omit values such as `AbortSignal`, auth context, locale, or - other request-scoped inputs unless sharing one result is correct. + `getOrLoad()` loader. Concurrent same-key misses share the leader's execution + unless the resolved policy explicitly sets `coalesce: false`. Do not omit + values such as `AbortSignal`, auth context, locale, or other request-scoped + inputs unless both sharing in-flight work and reusing a settled cache value + are correct. - **Methods need a receiver.** Pass `obj.method.bind(obj)` or `(...args) => obj.method(...args)`; a bare `obj.method` reference loses `this`. @@ -220,7 +227,7 @@ Instance-wide behavior is set through the `DialCache` constructor: | `DialCacheConfig` option | Default | Description | | --- | --- | --- | | `namespace` | `"urn"` | Logical cache namespace and first key component. | -| `redis` | none | `{ client, readTimeoutMs?, serializer? }`; enables the [remote layer](redis.md). Remote reads default to a 50 ms deadline. | +| `redis` | none | `{ client, readTimeoutMs?, serializer?, compression? }`; enables the [remote layer](redis.md). Remote reads default to a 50 ms deadline, and Redis payload compression defaults to zstd level 3 at 4,096 serialized bytes. | | `localMaxSize` | `10_000` | Global process-local entry cap. `0` disables process-local storage. Must be a nonnegative safe integer. | | `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the operation's `defaultConfig`; `null` applies no overrides. | | `shadowMaxInFlight` | `1` | Maximum scheduled or running shadow jobs per instance. Must be a positive safe integer. There is no queue; excess jobs are dropped and measured. | @@ -228,10 +235,10 @@ Instance-wide behavior is set through the `DialCache` constructor: | `logger` | `console` | Receives operational cache failures and opted-in confirmed shadow mismatch warnings through `debug`, `warn`, and `error`. | Per-invocation policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` -maps keyed by `CacheLayer.LOCAL` and `CacheLayer.REMOTE`, a `requestLocal` -boolean, an optional `remoteReadTimeoutMs`, and an optional `shadow` group. -The root-exported `ShadowConfig` type defines that group's independent `ramp` -and default-off `logMismatches` leaves. +maps keyed by `CacheLayer.LOCAL` and `CacheLayer.REMOTE`, `requestLocal` and +`coalesce` booleans, an optional `remoteReadTimeoutMs`, and an optional +`shadow` group. The root-exported `ShadowConfig` type defines that group's +independent `ramp` and default-off `logMismatches` leaves. ### Baseline and overlay precedence @@ -246,7 +253,8 @@ runtime field -> defaultConfig field -> DialCache disabled baseline ``` The disabled baseline sets `requestLocal` to `false`, leaves the process-local -and remote TTLs unset, and leaves `shadow` absent. +and remote TTLs unset, and leaves `shadow` absent. Coalescing defaults to +`true`, but no flight exists while every cache layer is inactive. Either serving layer is disabled by policy when it has no effective TTL. With an effective TTL but no effective ramp, that layer defaults to a 100% ramp. @@ -264,24 +272,29 @@ runtime remoteReadTimeoutMs This value bounds how long DialCache waits for an active Redis or Valkey read. It can be tuned per use case at runtime, but it cannot be disabled. -`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined`, so the -overlay can distinguish omission from an explicit `false`. Its effective value -still defaults to `false` after resolution. +`DialCacheKeyConfig` preserves omitted `requestLocal` and `coalesce` leaves as +`undefined`, so the overlay can distinguish omission from an explicit +`false`. Their effective defaults are `false` for request-local memoization and +`true` for coalescing. A provider result of `null`, or a defensive `undefined`, applies no overrides. An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the baseline. -Overlay merging is sparse at each leaf. The local and remote entries inside -`ttlSec` and `ramp` merge independently, as do `shadow.ramp` and -`shadow.logMismatches`. For example, `shadow: { ramp: 0 }` disables inherited -shadow admission while preserving an inherited logging preference; -`shadow: { logMismatches: false }` suppresses warnings without changing the -inherited shadow cohort. +Overlay merging is sparse at each leaf. Top-level `requestLocal`, `coalesce`, +and `remoteReadTimeoutMs` leaves merge independently. The local and remote +entries inside `ttlSec` and `ramp` also merge independently. + +The `shadow.ramp` and `shadow.logMismatches` leaves follow the same rule. For +example, `shadow: { ramp: 0 }` disables inherited shadow admission while +preserving an inherited logging preference; `shadow: { logMismatches: false }` +suppresses warnings without changing the inherited shadow cohort. Use explicit values to replace inherited policy: - `requestLocal: false` disables request-local caching; +- `coalesce: false` gives each caller its own active layer reads, fallback + deadline, fallback execution, and cache writes; - a process-local or remote ramp of `0` disables that serving layer; - `shadow: { ramp: 0 }` disables new shadow work; and - `DialCacheKeyConfig.disabled()` turns request-local and shadow work off and @@ -291,10 +304,14 @@ The remote serving and shadow cohorts are independent. A remote ramp of `0` does not override an inherited nonzero `shadow.ramp`; set both to `0` when the runtime policy must stop new invocation-driven Redis reads and fills. -`DialCacheKeyConfig.disabled()` returns the complete overlay explicitly: -`requestLocal: false`, both serving ramps at `0`, `shadow.ramp: 0`, and -`shadow.logMismatches: false`. Its `ttlSec` map is empty, so inherited TTLs -remain available for a later ramp-up but inactive under this overlay. The kill +`DialCacheKeyConfig.disabled()` returns the complete cache-path overlay +explicitly: `requestLocal: false`, both serving ramps at `0`, +`shadow.ramp: 0`, and `shadow.logMismatches: false`. It intentionally leaves +`coalesce` unset. + +Its `ttlSec` map is empty, so inherited TTLs remain available for a later +ramp-up but inactive under this overlay. If runtime policy ramps a layer back +up, coalescing is on again unless another leaf explicitly opts out. The kill switch does not cancel already-admitted work or disable explicit maintenance operations such as `invalidateRemote()`. @@ -308,13 +325,15 @@ whenever `getOrLoad()` is invoked: - serving ramps and `shadow.ramp` must be finite percentages in the inclusive range `0` through `100`; - layer maps and `shadow` must be objects; -- `requestLocal` and `shadow.logMismatches` must be booleans when present; and +- `requestLocal`, `coalesce`, and `shadow.logMismatches` must be booleans when + present; and - remote-read deadlines must be positive safe integers no greater than 2,147,483,647 milliseconds. -Invalid instance `redis.readTimeoutMs` values throw during `DialCache` -construction, as does an invalid `shadowMaxInFlight`. Invalid defaults are -rejected when `cached()` registers a definition or `getOrLoad()` is invoked. +Invalid instance `redis.readTimeoutMs` or `redis.compression` values throw +during `DialCache` construction, as does an invalid `shadowMaxInFlight`. +Invalid defaults are rejected when `cached()` registers a definition or +`getOrLoad()` is invoked. `null`, zero, fractional, non-finite, string, and larger timeout values are invalid; remote reads have no unbounded escape hatch. @@ -333,8 +352,9 @@ valid default leaves: Invalid leaves also record a `config_resolution` error, distinguishing provider garbage from an intentional ramp-down. A malformed runtime config object, -layer-map or `shadow` shape, `requestLocal`, or `remoteReadTimeoutMs` value fails -config resolution for the whole invocation. DialCache records +layer-map or `shadow` shape, `requestLocal`, `coalesce`, or +`remoteReadTimeoutMs` value fails config resolution for the whole invocation. +DialCache records `config_resolution`, marks the no-layer path `config_error`, and runs the fallback without a Redis read or write. @@ -437,9 +457,12 @@ eligible key. `shadow.logMismatches` separately opts confirmed mismatches into byte-capped JSON warning fields; it does not enable shadow work and defaults to `false`. Review the data-handling contract before turning it on. -Shadowing requires a valid remote TTL, invalidation tracking, and a metrics -adapter with the shadow outcome hook. It can validate a served Redis hit or -exercise Redis while the remote serving ramp excludes the key. See +Shadowing requires a valid remote TTL and a metrics adapter with the shadow +outcome hook. Tracked and untracked Redis operations are both eligible and +keep their normal read and write mode. + +Shadow work can validate a served Redis hit or exercise Redis while the remote +serving ramp excludes the key. See [Shadow validation and Redis bootstrap](shadow-validation.md) for eligibility, clean-miss filling, deadlines, capacity, and rollout guidance. @@ -448,6 +471,29 @@ defaults to Node strict deep equality and receives borrowed decoded-cache and source values. A custom comparator must be synchronous, deterministic, side-effect-free, non-mutating, and bounded. +### Coalescing policy + +Coalescing is on unless the resolved policy explicitly sets +`coalesce: false`. The switch covers both request-local and instance-scoped +process flights. + +With it off, concurrent same-key callers each perform their own active layer +reads, receive a full independent remote-read and fallback budget, run their +own loader after a miss, and attempt their own writes. +Settled request-local memoization still serves later sequential calls. +Process-local and untracked Redis writes remain last-writer-wins, while tracked +Redis writes retain their watermark fence. + +Opt out when callers sharing one identity must not inherit another caller's +loader failure, timeout, or cancellation behavior. Doing so reintroduces +same-key fan-out to dependencies. + +It also suppresses coalesced-follower metrics and keeps those calls out of +`getCoalescingState()`; each caller emits its own request, miss, latency, and +error observations. See +[Coalescing and async liveness](coalescing.md) for flight scope, deadlines, +shadow scheduling, and observability details. + ### Provider key input `cacheConfigProvider` receives the fully constructed, read-only `DialCacheKey` @@ -483,6 +529,55 @@ the root exports: The namespace and hash-tag components reject `{` and `}` as described under [Identity rules](#identity-rules). +## Redis payload compression + +`RedisConfig.compression` is instance-wide write policy for the remote layer. +It is enabled by default when Redis is configured: + +```ts +import { DialCache, type CompressionConfig } from "dialcache"; + +const compression: CompressionConfig = { + thresholdBytes: 4_096, + level: 3, +}; + +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + compression, + }, +}); +``` + +`thresholdBytes` must be a positive safe integer and defaults to `4_096`. +`level` must be an integer from `1` through `22` and defaults to `3`. +Passing `false` disables compression for new writes; `null`, other non-object +values, and invalid leaves throw during `DialCache` construction. Compression +is static instance configuration rather than per-use-case runtime policy. + +DialCache compresses a serialized payload only when it meets the threshold and +the zstd frame plus its marker is smaller than the raw stored form. Reads +always decode marked payloads, even when writes use `compression: false`, so +turning compression off does not orphan entries already written compressed. + +Raw binary serializer output beginning with an envelope byte is escaped on +every write, including when compression is disabled. + +Compression and decompression run synchronously on the Node.js event loop. +The exact package engine range is `>=22.15.0 <23.0.0 || >=23.8.0` so +`node:zlib` exposes zstd. + +Decompressed payloads are capped at 512 MiB, and the write side refuses to +compress values above the same ceiling. Start with the default level, watch +compression duration and ratio metrics, and treat higher levels as a +latency-sensitive production change. + +See [Redis payload compression](redis.md#compression) for the exact envelope, +mixed-version rollout and rollback behavior, binary-serializer migration, and +failure semantics. See [Observability](observability.md#compression-metrics) +for the bounded outcomes and pre- versus post-compression measurements. + ## Request-local cache Set `requestLocal: true` to memoize resolved values for the lifetime of the diff --git a/docs/invalidation.md b/docs/invalidation.md index ee80ebc..e964d3f 100644 --- a/docs/invalidation.md +++ b/docs/invalidation.md @@ -96,8 +96,11 @@ it. ## Read and write behavior -A tracked Redis value whose Redis-created timestamp is older than or equal to -the watermark is treated as stale and refreshed through fallback. +A tracked read obtains the value and watermark in one atomic `MGET`. Bundled +cluster adapters explicitly route it to the slot primary; a standalone +node-redis client must already target the authoritative endpoint. A readable +frame whose Redis-stamped creation time is older than or equal to the watermark +is treated as stale and refreshed through fallback. `invalidateRemote(keyType, id, futureBufferMs)` sets the watermark to the greater of: @@ -107,12 +110,15 @@ greater of: While that future window is active: -1. A tracked Redis read treats the covered value as a miss. +1. A tracked Redis read receives the covered value and watermark, then treats + the value as a miss. 2. The invocation runs its fallback. -3. If that fallback reaches the tracked Redis write before the window ends, - Redis rejects the write. -4. DialCache also suppresses the corresponding process-local population. -5. The fallback value still returns to its caller. +3. DialCache serializes and optionally compresses the fallback value, then a + native `SET` writes the complete payload as an unreadable placeholder. +4. A small stamp script compares Redis time with the watermark. If the window + is still active, it unlinks the placeholder and refuses publication. +5. DialCache suppresses the corresponding process-local population, while the + fallback value still returns to its caller. Request-local memoization remains unconditional. A ramped-out invocation without selected shadow work does not consult the watermark and is not fenced @@ -129,6 +135,27 @@ and process-local publication. This differs from a normal tracked miss, which can attempt the fenced Redis write. Untracked fallbacks may still populate process-local cache, and request-local memoization remains unconditional. +### Invalidated payload transfer and cleanup + +The atomic `MGET` transfers the complete Redis frame before the adapter can +compare its timestamp with the watermark. Large invalidated values can +therefore consume network bandwidth—and can repeatedly exceed the remote-read +deadline—even though DialCache will not serve them. + +A successful fallback that reaches the tracked stamp while the fence is active +partially mitigates this: its placeholder `SET` replaces the stale frame and +the stamp script unlinks the placeholder. Later reads then avoid transferring +the old payload. + +A read error or timeout skips the write entirely, so it cannot perform this +cleanup. A fallback or write failure can likewise leave cleanup for a later +successful attempt or the value TTL. + +The cleanup is not free. Every fenced write sends and temporarily stores the +complete serialized, possibly compressed payload before removing it. Include +that network transfer, Redis allocation, replication or AOF work, and stamp +round trip when estimating the load created by an oversized future buffer. + ### Shadow reads and fills [Shadow mode](shadow-validation.md) uses the same tracked protocol. A sampled @@ -211,9 +238,10 @@ value based on measured or conservatively bounded timings. Size it to cover: - the full remaining tail of any fallback that may already have observed the pre-mutation value; - `serializer.dump`; -- Redis client queue and network latency; -- Lua script execution; -- the Redis write itself; and +- synchronous compression or raw-payload escaping; +- Redis client queue and network latency for the full placeholder payload; +- the native placeholder `SET` and the tracked stamp script, including their + ordered dispatch and settlement; and - a safety margin. Include the remaining lifetime of any sampled shadow fill based on a source @@ -236,6 +264,14 @@ callers. ## Failure behavior and telemetry +The bundled adapters dispatch invalidation with `EVALSHA` and retry a rejected +dispatch once with the script source through `EVAL`. Because its monotonic +update only advances the watermark and widens its lifetime, duplicate +execution after an ambiguous first result is safe. A successful recovery is +internal to the adapter and produces no DialCache error or retry metric. See +[Mutation retries and ambiguity](redis.md#mutation-retries-and-ambiguity) for +adapter-specific error handling. + For a valid buffer, DialCache invokes the configured invalidation metric hook with `layer="remote"` before it checks the Redis prerequisite. @@ -246,6 +282,12 @@ invokes the configured error metric hook with `useCase="watermark"`, the original error. Logger and metrics callback failures are isolated and cannot replace that rejection. +A surfaced mutation failure is ambiguous: Redis may have advanced the +watermark before the client lost the reply. Do not interpret the rejection as +proof that nothing executed. Repeating `invalidateRemote()` after the source +mutation has committed is safe and advances or preserves the fence, but may +extend the future miss window. + ## In-memory layers remain local Targeted invalidation is remote-only. `invalidateRemote` does not evict existing diff --git a/docs/maintainers.md b/docs/maintainers.md index 7045011..f67dd86 100644 --- a/docs/maintainers.md +++ b/docs/maintainers.md @@ -18,9 +18,14 @@ uses Testcontainers and requires a working Docker-compatible container runtime for Redis, Valkey, and Redis Cluster. CI runs development and integration checks on Node.js 24, then switches to the -declared minimum Node.js 22.0.0 to test the packed package. Keep the consumer -floor separate from the development runtime so a new dependency or emitted -syntax cannot silently raise the published requirement. +exact 22.x consumer floor, Node.js 22.15.0, to prove both the packed package and +`node:zlib` zstd support. The published engine range is +`>=22.15.0 <23.0.0 || >=23.8.0`, because Node.js 23.0 through 23.7 do not expose +the required zstd API. + +Keep the consumer floor separate from the development runtime so a dependency, +emitted syntax, or runtime API cannot silently raise the published +requirement. Before changing a compatibility-sensitive surface, identify and extend the corresponding packed, unit, and integration assertions: @@ -32,17 +37,29 @@ corresponding packed, unit, and integration assertions: - deterministic serving- and shadow-ramp assignment, whose independent cohorts must not reshuffle across releases, plus nested shadow-policy snapshot, overlay, validation, and legacy `shadowRamp` rejection; -- the binary Redis frame, Lua arguments and reply domains, tracked - read/write/invalidation semantics, mixed-version serializer behavior, and the - ownership and immutability contract for retained string and `Buffer` - payloads; +- `coalesce` omission defaulting to enabled, sparse boolean overlays, explicit + opt-out in both request and process scopes, independent deadlines and writes, + settled request-local reuse, and malformed-value fail-open behavior; +- native `GET`/`MGET` reads, native untracked `SET` writes, and the ordered + tracked placeholder-`SET` plus stamp-script pair, including exact frame + encoders, script reply domains, the root-exported placeholder-loss error, + wrong-type behavior, Redis Cluster routing, and removed read/write-script + exports; +- tracked invalidation plus tracked and untracked shadow behavior, + mixed-version serializer behavior, and the ownership and immutability + contract for retained string and `Buffer` payloads; +- default-on zstd configuration and validation, binary envelope collisions, + decompression caps, raw fallback, mixed-version upgrades and rollbacks, + first-party and optional custom-adapter metrics, and the exact Node.js floor; - rejection and bounded error telemetry when `invalidateRemote()` is called without a configured Redis client; - shadow confirmation, clean-miss fill, capacity, deadline, detached work, and payload-release behavior, plus default-off confirmed-mismatch logging and its byte-capped native-JSON detail fields; - exhaustive public unions and packed exports, including `MetricLayer`, - `ShadowValidationOutcome`, `ShadowComparator`, and `ShadowConfig`; and + `ShadowValidationOutcome`, `ShadowComparator`, `ShadowConfig`, + `CompressionConfig`, compression metric types, and Redis protocol error + classes; and - bounded metrics names, labels, reasons, error categories, scopes, outcomes, units, and observer isolation from synchronous throws and rejected thenables. @@ -89,6 +106,38 @@ Override its work sizes with: - `DIALCACHE_BENCH_ITERATIONS`; and - `DIALCACHE_BENCH_FANOUT`. +## Redis write benchmark + +With a Redis server reachable at `REDIS_URL` (default +`redis://127.0.0.1:6379`), run: + +```bash +corepack pnpm benchmark:redis-write +``` + +The command builds `dist`, then measures eight sequential configurations: +tracked and untracked writes at 100 B, 10 KiB, 100 KiB, and 1 MiB. It reports +server-side command time per write from `INFO commandstats`, plus client-side +p50 and p95 latency. For tracked writes, the `EVALSHA` entry envelopes the +stamp script's internal command cost. + +This benchmark is a maintainer diagnostic and is not included in the published +package. It deliberately has no semantic assertion or timing threshold: +absolute results depend on the machine, Redis engine, payload, and ambient +load. + +Run it only against a dedicated disposable or development Redis. It writes +fixed `benchmark:write:*` keys and executes `CONFIG RESETSTAT` before every +sample, so the client needs that permission and the command erases the +server's accumulated command statistics. The script calls the semantic +adapter's `write()` method directly with prebuilt payloads; it measures neither +serializer nor compression cost. + +Compare implementations only with fresh alternating samples in the same +environment, and preserve correctness coverage in unit, packed-package, and +live integration tests. Scale every iteration count with +`DIALCACHE_BENCH_WRITE_SCALE`. + ## Releasing Publishing starts by manually running the `Release` workflow from current @@ -97,13 +146,18 @@ Publishing starts by manually running the `Release` workflow from current After the package checks pass, Semantic Release selects the next version from Conventional Commits since the highest stable `vX.Y.Z` tag: -- breaking changes bump major; +- while the package is pre-1.0, breaking changes bump minor and retain their + `BREAKING CHANGE:` footers for full release notes; - `feat` bumps minor; and - every other normal PR-title type bumps patch. Patch types are `fix`, `perf`, `docs`, `style`, `refactor`, `test`, `build`, `chore`, `ci`, and `revert`. The highest required bump wins. +Major bumps resume when the project cuts 1.0.0. `release.config.mjs` implements +this policy; change it and this guide together so the documented release table +cannot drift from automation. + The workflow opens a `release: ` pull request whose only change is the matching `package.json` version. `release` is a reserved Conventional Commit type configured not to request another release, so the version-control commit diff --git a/docs/observability.md b/docs/observability.md index 7c378b6..63f3744 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -64,10 +64,14 @@ The names below exclude the optional caller-selected prefix: | `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | | `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by request-local or process scope | | `dialcache_shadow_validation_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `outcome` | Terminal outcomes for sampled Redis shadow jobs | +| `dialcache_compression_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `outcome` | Bounded Redis payload compression and decompression outcomes | | `dialcache_get_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | | `dialcache_fallback_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | | `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | -| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | +| `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes, before compression | +| `dialcache_stored_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Prepared Redis payload size in bytes, after compression and escaping | +| `dialcache_compression_ratio_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Compressed-to-original payload size ratio for compressed writes | +| `dialcache_compression_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Compression and decompression latency in seconds | The disabled reasons are: @@ -93,15 +97,16 @@ The `layer` label is: - `request_local`; - `local`, meaning process-local; - `remote`; -- `remote_shadow` for Redis reads, fills, serialization, and payload sizes - performed by detached shadow jobs; or +- `remote_shadow` for Redis reads, fills, serialization, compression, and + payload sizes performed by detached shadow jobs; or - `noop` for disabled-context, key-construction, and config-provider failures where no cache layer was reached. The bounded `scope` label on `dialcache_coalesced_counter` distinguishes `request_local` from `process`. `scope="process"` coordinates calls only within one `DialCache` instance; separate instances in the same process do not share -in-flight state. +in-flight state. A use case with `coalesce: false` emits no coalesced counter; +each caller instead emits its own request, miss, duration, and error metrics. ### Shadow outcomes @@ -115,8 +120,8 @@ outcomes through `dialcache.shadow.count`: | `mismatch` | They differed, and a confirmation read found the original Redis payload unchanged. | | `superseded` | They differed, but the Redis payload changed or disappeared before confirmation. | | `filled` | A clean shadow miss was populated successfully. | -| `fill_blocked` | An invalidation watermark blocked a clean-miss fill. | -| `fill_error` | Serializing or writing a clean-miss fill failed. | +| `fill_blocked` | An invalidation watermark blocked a tracked clean-miss fill; compliant untracked writes do not produce it. | +| `fill_error` | Preparing the payload (serialization or compression) or writing a clean-miss fill failed. | | `redis_error` | The initial detached Redis read failed. | | `source_error` | The source-of-truth read failed. | | `deserialization_error` | The retained Redis payload could not be deserialized for comparison. | @@ -131,6 +136,61 @@ keeps detached work separate from caller-serving `layer="remote"` telemetry. See [Shadow validation](shadow-validation.md) for the read, confirmation, fill, and deadline semantics behind these outcomes. +### Compression metrics + +Compression telemetry is bounded and uses `layer="remote"` for caller-serving +work or `layer="remote_shadow"` for detached shadow work. + +Write-side outcomes are: + +- `compressed`: zstd plus its envelope was smaller and selected for the + prepared Redis payload; +- `below_threshold`: the serialized payload did not reach the configured + threshold; +- `not_smaller`: compression ran, but the marked result was not smaller than + the raw stored form; and +- `write_over_limit`: the serialized value exceeded the 512 MiB decompression + ceiling and was kept raw for the attempted write. This is a capacity signal, + not an error. + +Read-side outcomes are: + +- `decompressed`: a marked zstd payload was restored; +- `fallback_raw`: a marked payload was not valid zstd and was passed unchanged + to the serializer; and +- `read_over_limit`: decompression would exceed the 512 MiB ceiling, so the + stored bytes were passed unchanged to the serializer. Treat this as a + corruption or integrity signal. + +Raw reads do not emit a compression outcome. With `compression: false`, new +writes are still escaped when necessary but emit no compression outcome; reads +continue to report marked values because disabling writes does not disable +decoding. + +`dialcache_size_histogram` measures serializer output before compression and is +the distribution to use when selecting `thresholdBytes`. +`dialcache_stored_size_histogram` measures the prepared bytes after compression +or binary-envelope escaping. DialCache records it before the shadow deadline +gate and before calling the Redis client, so it is not proof that a write was +dispatched or succeeded. The ratio histogram is emitted when compression +selects the smaller representation, at the same pre-write stage. + +Compression duration is observed when zstd runs and produces either +`compressed` or `not_smaller`; decompression duration is observed for each +marked payload that produces a read-side outcome. + +A zstd exception while preparing a write records `error="compression"` and +the cache write fails open. Decompression rejects neither the cache call nor +the observer path directly: an unreadable payload reaches the configured +serializer, whose rejection follows the existing refreshable-miss path and +records `serialization_load`. + +zstd work is synchronous on the Node.js event loop. Use the duration, ratio, +and pre/post-size series together when changing the threshold or level; a good +space ratio does not make an event-loop stall acceptable. See +[Redis payload compression](redis.md#compression) for the envelope, limits, +and mixed-version rollout contract. + ### Confirmed mismatch warnings Shadow metrics remain bounded and contain no cache ids or values. A use case can @@ -165,7 +225,7 @@ import { DialCache } from "dialcache"; import { createDatadogDialCacheMetrics } from "dialcache/datadog"; const dogStatsD = new StatsD({ - host: process.env.DD_AGENT_HOST, + host: process.env.DD_AGENT_HOST ?? "127.0.0.1", globalTags: { service: "users-api", env: process.env.DD_ENV ?? "development", @@ -209,8 +269,8 @@ hosts. Enable the desired distribution percentiles and aggregations in Datadog. Choose `"histogram"` when host-level histogram aggregation matches the existing -Datadog setup. The choice applies uniformly to all four duration and size -metrics. Both modes produce Datadog custom metrics. +Datadog setup. The choice applies uniformly to every duration, size, and ratio +observation emitted by the adapter. Both modes produce Datadog custom metrics. Distribution volume scales with unique tag-value combinations. Datadog counts five baseline aggregations per combination; enabling percentile aggregations @@ -257,10 +317,14 @@ and bytes without unit conversion: | `dialcache.invalidation.count` | Count | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | | `dialcache.coalesced.count` | Count | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests by sharing scope | | `dialcache.shadow.count` | Count | `cache_namespace`, `use_case`, `key_type`, `outcome` | Terminal outcomes for sampled Redis shadow jobs | +| `dialcache.compression.count` | Count | `cache_namespace`, `use_case`, `key_type`, `layer`, `outcome` | Bounded Redis payload compression and decompression outcomes | | `dialcache.get.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache get latency in seconds | | `dialcache.fallback.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Elapsed time until the wrapped fallback settles or timeout rejection is delivered | | `dialcache.serialization.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency in seconds | -| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | +| `dialcache.serialization.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes, before compression | +| `dialcache.stored.size` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Prepared Redis payload size in bytes, after compression and escaping | +| `dialcache.compression.ratio` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Compressed-to-original payload size ratio for compressed writes | +| `dialcache.compression.duration` | Distribution or histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Compression and decompression latency in seconds | Client throws and rejected returned thenables are isolated by DialCache's fire-and-forget observer boundary. Buffered transport failures that happen @@ -279,9 +343,10 @@ thrown value's class or `Error.name`: | `config_resolution` | Runtime or layer configuration validation or resolution failed | | `cache_read` | A process-local read or non-timeout remote read failed | | `cache_read_timeout` | A remote read exceeded its effective DialCache deadline | -| `cache_write` | A process-local or remote cache write failed | +| `cache_write` | A process-local or remote cache write failed; native tracked writes include the observable lost-placeholder race described below | | `serialization_load` | Deserializing a Redis payload failed | | `serialization_dump` | Serializing a value for Redis failed | +| `compression` | zstd compression failed while preparing a Redis write | | `invalidation` | Writing an invalidation watermark failed | | `fallback` | The source loader failed or exceeded its DialCache deadline | | `unknown` | Reserved for a future failure site that cannot be classified otherwise | @@ -301,6 +366,18 @@ the wait. Coalesced followers do not multiply the timeout error. Deadline details remain out of labels and are available on the logged `RedisReadTimeoutError`. +A tracked native write first stores an unreadable placeholder and then stamps +that exact placeholder through the small mutation script. If another write +overwrites it, it expires, or a watermark-fenced write removes it before the +stamp, the adapter raises the root-exported +`DialCacheRedisPlaceholderLostError`. DialCache records one +`error="cache_write"`, suppresses publication of that write, and logs a warning. + +Same-key write contention can therefore create a benign, self-healing floor of +these errors around hot-key expiry. Keep the metric bounded, use the error +class in structured logs or direct adapter calls to distinguish the case, and +rate-limit the warning sink when that contention is expected. + Raw thrown values, error names, messages, cache ids, arguments, and Redis keys are never included in labels. When DialCache logs a cache-plumbing failure, the raw details remain available through the configured logger; not every metric @@ -326,18 +403,29 @@ Implement `DialCacheMetricsAdapter` and pass it through | `invalidation(labels)` | yes | One explicit remote invalidation call. | | `coalesced(labels)` | no | One follower that joined request-local or process-scoped work. | | `shadowValidation(labels)` | no | One terminal sampled-shadow outcome. This hook must be implemented for shadow jobs to execute. | +| `compression(labels)` | no | One bounded compression or decompression outcome. | | `observeGet(labels, seconds)` | yes | Cache-read duration in seconds. | | `observeFallback(labels, seconds)` | yes | Fallback duration in seconds. | | `observeSerialization(labels, seconds)` | yes | Serializer dump/load duration in seconds. | -| `observeSize(labels, bytes)` | yes | Serialized remote payload size in bytes. | +| `observeSize(labels, bytes)` | yes | Serialized remote payload size in bytes, before compression. | +| `observeStoredSize(labels, bytes)` | no | Prepared remote payload size in bytes, after compression and escaping; emitted before client dispatch. | +| `observeCompressionRatio(labels, ratio)` | no | Compressed-to-original size ratio when compression selects the prepared representation. | +| `observeCompression(labels, seconds)` | no | Compression or decompression duration with `operation="compress"` or `operation="decompress"`. | The root package exports `DialCacheMetricsAdapter` and every associated label, reason, error-kind, layer, scope, and shadow-outcome type, including -`ShadowValidationMetricLabels` and `ShadowValidationOutcome`. +`ShadowValidationMetricLabels`, `ShadowValidationOutcome`, +`CompressionMetricLabels`, `CompressionOperationMetricLabels`, and +`CompressionOutcome`. `shadowValidation` remains optional so existing custom adapters keep compiling, but DialCache does not admit shadow work when the configured adapter omits it. The Prometheus and Datadog adapters implement the hook. +The compression hooks are also optional for source compatibility with existing +custom adapters. They control observation only: omitting them does not disable +compression or decompression. The Prometheus and Datadog adapters implement +all four hooks. + Metrics and logger methods are typed `void` and invoked as fire-and-forget observers. DialCache also defensively consumes, but never awaits, a thenable returned at runtime. diff --git a/docs/redis.md b/docs/redis.md index e74e92a..4b79810 100644 --- a/docs/redis.md +++ b/docs/redis.md @@ -7,9 +7,10 @@ Redis Cluster. The application creates, connects, configures, drains, and closes the underlying client. DialCache borrows a client-independent `DialCacheRedisClient` adapter and does not own the connection lifecycle. -Sampled non-serving Redis reads and fills use the same adapter and tracked -protocol. See [Redis shadow validation](shadow-validation.md) for eligibility, -comparison, capacity, metrics, and rollout behavior. +Sampled non-serving Redis reads and fills use the same adapter and preserve the +operation's tracked or untracked mode. See +[Redis shadow validation](shadow-validation.md) for eligibility, comparison, +capacity, metrics, and rollout behavior. ## Install a client @@ -20,13 +21,13 @@ Choose one supported integration: npm install redis@~4.7.1 # or Valkey GLIDE -npm install @valkey/valkey-glide +npm install @valkey/valkey-glide@^2.0.0 ``` ## node-redis -Register DialCache's native scripts when creating the client, connect it, and -pass the DialCache-compatible adapter to `DialCache`: +Register DialCache's two mutation scripts when creating the client, connect +it, and pass the DialCache-compatible adapter to `DialCache`: ```ts import { createClient } from "redis"; @@ -37,7 +38,7 @@ import { } from "dialcache/node-redis"; const redisClient = createClient({ - url: process.env.REDIS_URL, + url: process.env.REDIS_URL ?? "redis://127.0.0.1:6379", scripts: dialcacheRedisScripts, disableOfflineQueue: true, commandsQueueMaxLength: 1_000, @@ -66,16 +67,23 @@ users should register the supplied scripts and wrap the connected client with instance-wide value or use `DialCacheKeyConfig.remoteReadTimeoutMs` for per-use-case static and runtime policy. +Use node-redis's promise-mode client; `legacyMode` is not supported. Treat +`dialcacheRedisScripts` as adapter wiring rather than a direct write API. Its +`dialcacheWriteTrackedStamp` method returns the raw `0 | 1 | 2` script reply; +direct callers must pass that reply through `resolveTrackedRedisWriteReply` +from `dialcache/redis-protocol` so reply `2` becomes a lost-placeholder error. + Local-only caching does not require a Redis client, but the explicit remote maintenance operation `invalidateRemote()` does. It rejects when Redis is not configured so a caller cannot mistake an absent watermark write for successful invalidation. See [Targeted invalidation](invalidation.md) for the complete contract. -The adapter computes each script's SHA, uses `EVALSHA`, and retries with `EVAL` -after `NOSCRIPT`. Its cluster client routes scripts by their first key and -performs that fallback on the selected shard. Tracked reads are deliberately -routed to primaries so a lagging replica cannot hide an invalidation watermark. +The registered scripts stamp tracked writes and advance invalidation +watermarks. Reads and untracked writes use native Redis commands. Node-redis +performs its normal script-cache recovery for the stamp script; the adapter's +additional invalidation recovery is described under +[Mutation retries and ambiguity](#mutation-retries-and-ambiguity). Deployments using tracked invalidation must also satisfy the [watermark durability](invalidation.md#watermark-durability) contract. @@ -110,19 +118,150 @@ const dialcache = new DialCache({ function shutdown(): void { // Drain cached calls and invalidations before releasing resources. - redisClient.dispose(); glideClient.close(); } ``` -DialCache uses the supplied namespace's `Script` constructor and -`Decoder.Bytes` value without importing a GLIDE runtime itself. Passing the same -module namespace that created the client prevents linked workspaces or -applications with another installed GLIDE version from mixing native script -handles. +DialCache uses the supplied namespace's `Batch`, `ClusterBatch`, client +constructors, and `Decoder.Bytes` value without importing a GLIDE runtime +itself. Passing the same module namespace that created the client prevents +linked workspaces or applications with another installed GLIDE version from +mixing native objects. + +Pass a direct `GlideClient` or `GlideClusterClient` instance. Wrappers should +implement `DialCacheRedisClient` directly. The returned adapter is stateless, +owns no script handles, and needs no disposal; the application closes the +underlying GLIDE client after its work drains. + +## Bundled Redis operations + +The node-redis and GLIDE adapters preserve the same semantic protocol while +using each client's native command and routing APIs. + +### Reads + +- An untracked read is one native `GET`. +- A tracked read is one atomic `MGET valueKey watermarkKey`. Cluster adapters + explicitly route it to the slot primary, even when replica reads are enabled, + so replica lag cannot hide an invalidation watermark. +- Missing, short, unsupported-version, or placeholder frames are clean misses. + A tracked read also misses when the watermark is missing or malformed, or + when `createdAt <= watermark`. + +The value and watermark must share a Redis Cluster slot; DialCache's generated +tracked keys do. Redis returns the complete value before the adapter compares +its timestamp with the watermark. A large invalidated value can therefore use +network bandwidth on every attempted read until a successful fallback write +removes it or its TTL expires. See +[Invalidated payload transfer and cleanup](invalidation.md#invalidated-payload-transfer-and-cleanup). + +GLIDE standalone sends tracked `MGET` through a one-command non-atomic batch +so the client routes it to the primary instead of applying its ordinary +one-key read preference. `MGET` itself remains the single atomic snapshot; the +batch is deliberately non-transactional and does not consume caller-owned +`WATCH` state. + +Node-redis standalone sends `MGET` to the endpoint the application configured; +the adapter cannot discover or reroute a standalone replica connection. Point +that client at the authoritative primary when relying on tracked invalidation. + +DialCache keys must remain application-owned strings. Native Redis type rules +are intentionally visible: `GET` rejects a wrong-type untracked value, while +`MGET` returns a missing member for a wrong-type tracked value or watermark. +A wrong-type tracked value can be replaced by the fallback write when its +watermark is valid. + +A wrong-type or malformed watermark makes reads miss, then causes the stamp to +fail after the placeholder `SET`; repeated calls therefore fail open and reload +until that watermark state is repaired. A valid-version frame with an +unsupported payload encoding is a typed payload error rather than a miss. + +### Writes + +An untracked write is one native command: + +```text +SET valueKey frame PX cacheTtlMs +``` -The GLIDE adapter uses GLIDE's native script lifecycle and byte decoder. GLIDE -routes scripts from their declared keys. +Its frame carries an informational client-clock timestamp. Untracked reads do +not consult that timestamp. + +A tracked write uses two commands, ordered on one connection without +`MULTI`/`EXEC`: + +1. `SET` writes the complete serialized payload in an unreadable version-0 + placeholder with a fresh nonce and the value TTL. +2. `WRITE_TRACKED_STAMP_SCRIPT` verifies that exact nonce, reads Redis time and + the watermark, and either promotes the placeholder to a readable frame, + unlinks it when the watermark fence is active, or reports that the + placeholder was lost. + +The nonce prevents a delayed stamp from publishing another writer's value. +The placeholder is a deliberate fail-safe: an interleaved or failed stamp is a +miss, not an unstamped cache hit. Because its `SET` replaces the prior frame, a +tracked write can briefly make a previously readable key miss while the stamp +settles. + +The pair is non-transactional so it does not consume caller-owned Redis +`WATCH` state. The bundled adapters enqueue or batch the pair in order. A +watermark fence returns `false`; DialCache returns the fallback value and does +not publish it process-locally. + +A missing, overwritten, or expired placeholder throws +`DialCacheRedisPlaceholderLostError`. Ordinary cached calls absorb that error +through the fail-open cache-write path, so same-key write contention can +produce benign bounded `cache_write` errors on hot keys. + +The adapter reports a failed `SET` as the write outcome even if the stamp also +settled. Because transport failures can be ambiguous, the `SET` may have +landed and the stamp may have promoted it despite the reported error. Never use +a cache-write rejection as proof that Redis was not mutated. + +### Mutation retries and ambiguity + +The bundled adapters dispatch invalidation with `EVALSHA`. If dispatch rejects, +they retry once with the monotonic, replay-safe script source through `EVAL`; +this also repairs a flushed script cache. A reply-domain violation is a +protocol error, not a retryable dispatch failure. If recovery fails, the retry +error surfaces. + +GLIDE attaches the original rejection as its `cause` when safe; node-redis does +not mutate the shared error objects it can use for disconnect failures. + +For the tracked stamp, node-redis uses its registered script's normal +`NOSCRIPT` recovery. GLIDE retries the stamp with `EVAL` only on `NOSCRIPT`, +because any other error may be an ambiguous result from a stamp that already +executed. The first tracked GLIDE write after a script-cache flush can +therefore pay one extra round trip. + +The retry is below the `DialCacheRedisClient` boundary. A successful recovery +is therefore not a DialCache error or retry metric, although Redis command +statistics can reveal the additional `EVAL`. + +Like any network mutation, a rejected write or invalidation can have executed +before the client reports failure. Do not add an outer `Promise.race` and +assume rejection proves non-execution; use finite client-native queue, +reconnect, dispatch, and response budgets. + +### Redis compatibility and ACLs + +The tracked stamp uses `UNLINK`, so the bundled protocol requires Redis 4 or a +compatible Valkey release. Redis Cluster deployments must allow multi-key +operations for keys in the same slot. + +At minimum, allow the client commands `GET`, `MGET`, `SET`, `EVALSHA`, and +`EVAL`. The scripts also invoke `TIME`, `GET`, `SET`, and `PTTL`; the tracked +stamp additionally invokes `PEXPIRE`, `UNLINK`, `GETRANGE`, and `SETRANGE`. +The bundled adapters do not require `SCRIPT LOAD`. + +Verify ACLs and proxy behavior before upgrading. A persistent stamp failure +still lets each placeholder `SET` replace the last readable value, while the +failed write suppresses process-local publication. Within one value-TTL +horizon, affected tracked keys can send all traffic to the source. Each lost +placeholder on a DialCache request path also records a bounded `cache_write` +error and emits a warning; size alerts and logger rate limits for expected +same-key contention. ## Lifecycle ownership @@ -137,8 +276,7 @@ The application owns the complete Redis lifecycle: Redis. 6. Drain or terminate client-native Redis work that may have outlived DialCache's caller-serving or shadow-read wait. -7. Dispose adapter-owned resources. -8. Close the underlying connection. +7. Close the underlying connection. DialCache has no `close()` or drain method. It never disposes or closes caller resources. @@ -152,13 +290,10 @@ shadow job can remain active during teardown. Stop new work before closing dependencies and use their native drain or termination controls. An already-dispatched shadow fill may have executed even if its outcome is lost. -The node-redis adapter owns no additional resources, so close the underlying -client after draining work. - -The GLIDE adapter owns five native `Script` handles but not the wrapped -connection. Call its idempotent `dispose()` after operations finish and before -closing GLIDE. Disposing while an adapter operation is in flight throws rather -than releasing a live script. A DialCache read timeout does not prove that the +Both bundled adapters are resource-free views over caller-owned clients. They +have no `dispose()` method and own no connection, batch, or script handle. +After DialCache and detached client work drain, close the underlying node-redis +or GLIDE client directly. A DialCache read timeout does not prove that the client-side invocation has settled. ## Remote-read deadlines and async liveness @@ -180,7 +315,7 @@ Each explicit value must be a positive safe integer no greater than Outside an enabled scope and on an earlier in-memory hit, DialCache creates no remote-read timer. A key ramped out of Redis serving creates no caller-serving timer, but an independently selected shadow job can create unreferenced timers -for its tracked `C0` and optional `C1` reads. +for its same-mode `C0` and optional `C1` reads. ### Caller-serving timeout and fail-open behavior @@ -205,8 +340,9 @@ publication because watermark safety was not established. Request-local memoization remains unconditional. Same-key callers in one request-local or process coalescing scope share the -leader's read, timer, and remaining budget. A later independent invocation may -start a new remote read even if the prior client operation is still settling. +leader's read, timer, and remaining budget. With `coalesce: false`, each caller +gets a full independent read budget and can start another remote read while a +prior client operation is still settling. The `fallbackTimeoutMs` timer is separate and starts only if and when the source loader begins. The remote-read timer covers neither config resolution, @@ -231,14 +367,17 @@ Custom adapters implement the complete client-independent read, write, and invalidate contract: ```ts -type Awaitable = T | Promise; +import type { + RedisCachePayload, + RedisInvalidationRequest, + RedisReadContext, + RedisReadRequest, + RedisWriteRequest, +} from "dialcache"; -interface RedisReadContext { - readonly timeoutMs: number; - readonly signal: AbortSignal; -} +type Awaitable = T | Promise; -interface DialCacheRedisClient { +interface DialCacheRedisClientContract { read( request: RedisReadRequest, context?: RedisReadContext, @@ -250,24 +389,32 @@ interface DialCacheRedisClient { #### `read` -- Return an operation-owned serialized `string` or `Buffer`, or `null` for a - miss. +- Decode native bulk-string replies with `decodeRedisFrame` or + `decodeTrackedRedisFrame` from `dialcache/redis-protocol`, or preserve their + exact behavior, and return the operation-owned serialized `string` or + `Buffer`, or `null` for a miss. - Keep the payload stable after settlement because DialCache can retain it for shadow work. An adapter that recycles response storage must return a dedicated `Buffer`. -- For a tracked request, compare the value timestamp and watermark atomically. - A missing watermark or a value at or behind it is a miss. +- For a tracked request, obtain the value and watermark atomically from one + authoritative primary snapshot. A missing or malformed watermark, or a + value at or behind it, is a miss. #### `write` -- Accept `cacheTtlMs` as a positive integer no greater than - `31_536_000_000` milliseconds (365 days), apply that TTL, and record server - time atomically. -- For a tracked request, create a missing baseline and retain it for at least - the value TTL plus one minute without shortening a longer or persistent - lifetime. -- Return `true` only when the value was written and `false` when publication is - rejected. +- Normalize `cacheTtlMs` with `ceilSupportedCacheTtlMs`; positive fractional + milliseconds round up, and the result may not exceed `31_536_000_000` + milliseconds (365 days). +- Use `encodeRedisFrame` for the one-command untracked path. For a tracked + request, preserve the exact placeholder-and-stamp behavior described above + with `encodeTrackedRedisPlaceholder`, `WRITE_TRACKED_STAMP_SCRIPT`, and + `resolveTrackedRedisWriteReply`. +- On a non-fenced tracked write, create a missing baseline watermark and retain + it for at least the value TTL plus one minute without shortening a longer or + persistent lifetime. +- Return `true` only when the value was published and `false` only when the + watermark fence rejected it. Surface a lost placeholder as + `DialCacheRedisPlaceholderLostError`, not as `false`. #### `invalidate` @@ -293,8 +440,8 @@ The bundled node-redis adapter forwards the signal in per-command options. This can remove queued work where supported, but aborting after dispatch cannot unsend a command or prove that Redis stopped executing it. -The GLIDE script API has no per-invocation signal, so a timed-out script -invocation may continue inside the adapter. Its configured +The GLIDE command API has no per-invocation signal, so a timed-out command may +continue inside the adapter. Its configured [`requestTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.BaseClientConfiguration.html) and [`advancedConfiguration.connectionTimeout`](https://glide.valkey.io/languages/nodejs/api/interfaces/BaseClient.AdvancedBaseClientConfiguration.html) @@ -440,7 +587,96 @@ serializers must treat payloads and values as borrowed and immutable, return independent values from repeated loads, and copy a Buffer before mutating it. See [Data ownership and custom integrations](shadow-validation.md#data-ownership-and-custom-integrations). -### Advanced wire protocol +## Compression + +Redis payload compression is enabled by default and configured once per +`DialCache` instance: + +```ts +const dialcache = new DialCache({ + redis: { + client: dialCacheRedisClient, + compression: { + thresholdBytes: 4_096, + level: 3, + }, + }, +}); +``` + +`thresholdBytes` must be a positive safe integer and defaults to 4,096 bytes. +`level` must be an integer from 1 through 22 and defaults to 3. Pass +`compression: false` to disable compression on future writes. This is an +instance-level write policy, not a per-use-case runtime ramp. + +DialCache measures the serializer output in bytes, then compresses it with +zstd only when it meets the threshold and the marked compressed form is +smaller than the raw stored form. Compression and decompression are +synchronous and run on the Node.js event loop. Benchmark representative value +sizes and zstd levels under production-like concurrency before lowering the +threshold or raising the level. + +Default-on compression requires zstd-capable `node:zlib`; DialCache validates +that support during construction. The package's supported Node.js range starts +at 22.15.0 in the 22.x line and excludes 23.0 through 23.7. The exact published +engine range is `>=22.15.0 <23.0.0 || >=23.8.0`. + +Reads always decode marked payloads, even when write-side compression is +disabled. That makes `compression: false` a safe way to stop producing new +compressed entries without orphaning existing ones. Binary serializer output +whose first byte is `0x00`, `0x01`, or `0x02` is also escaped on every write, +including when compression is disabled, so current readers can distinguish it +from the compression envelope exactly. + +### Size limits and failure behavior + +DialCache caps decompressed output at 512 MiB, matching Redis's value limit. +Serializer output above that cap is left raw for the Redis write rather than +compressed. + +If a marked value cannot be decompressed or would exceed the cap, DialCache +passes the original marked bytes to `serializer.load`; a validating serializer +will normally reject it and trigger the existing self-healing miss path. A +permissive custom serializer can instead accept those bytes, so monitor +`fallback_raw` and `read_over_limit` as payload-integrity signals rather than +assuming they always become misses. + +A write-side zstd exception records `error="compression"`, skips the Redis +write, and follows DialCache's fail-open cache-write path. The fallback result +still returns. Decompression outcomes are reported before serializer loading; +if loading then fails, the same read can also record `serialization_load`. + +Compression-aware metrics adapters can implement the optional `compression`, +`observeStoredSize`, `observeCompressionRatio`, and `observeCompression` +hooks. `observeSize` remains the serializer-output size before compression or +escaping; `observeStoredSize` measures the prepared payload afterward, before +the shadow deadline gate and Redis write. It does not prove that a write was +dispatched or succeeded. See +[Observability](observability.md#compression-metrics) for bounded outcomes and +the bundled Prometheus and Datadog metric names. + +### Rolling deployments and binary serializers + +Current readers accept frames from older releases. Older readers, however, do +not understand newly compressed or escaped payloads and will usually reject +them during deserialization, causing temporary fallback and refill churn in a +mixed deployment. For string and JSON serializers, a low-noise rollout is: + +1. deploy the new release everywhere with `compression: false`; +2. allow old readers to drain; and +3. enable compression in a later rollout. + +Apply the same consideration when rolling back while compressed entries still +exist. + +Before the escape envelope existed, arbitrary binary output could already +begin with an envelope marker. A legacy payload beginning with `0x00` followed +by `0x00`–`0x02`, or with `0x01`/`0x02` followed by a valid zstd stream, can be +misinterpreted by a current reader until it expires. If a custom serializer +can emit those prefixes, change the operation's `useCase` or other key-version +component for the migration. + +## Advanced wire protocol The core Redis boundary is the client-independent `DialCacheRedisClient` interface. It exchanges serialized values as `string | Buffer` and does not @@ -449,43 +685,58 @@ expose client-specific commands or wire encodings. The `dialcache/redis-protocol` entry point exports the exact bundled protocol building blocks: -- `READ_CACHE_SCRIPT` and `READ_TRACKED_CACHE_SCRIPT`; -- `WRITE_CACHE_SCRIPT` and `WRITE_TRACKED_CACHE_SCRIPT`; -- `INVALIDATE_CACHE_SCRIPT`; and -- `REDIS_FRAME_VERSION`, `REDIS_ENCODING_UTF8`, and - `REDIS_ENCODING_BINARY`. +- `decodeRedisFrame` and `decodeTrackedRedisFrame` for native read replies; +- `encodeRedisFrame`, `encodeTrackedRedisPlaceholder`, and the + `TrackedRedisPlaceholder` type for native writes; +- `ceilSupportedCacheTtlMs` for adapter-level write TTLs; +- `WRITE_TRACKED_STAMP_SCRIPT` and `INVALIDATE_CACHE_SCRIPT`; +- `resolveTrackedRedisWriteReply`; and +- `validateRedisSetReply` and `validateRedisScriptInvalidationReply`. -The scripts implement the atomic read, publication, invalidation, server-time, -and derived-watermark-lifetime behavior required above. Custom adapters can -throw these root-exported error classes: +The payload bytes inside the Redis frame are opaque to this adapter-level +protocol. Compression and escaping sit above it in DialCache core. Custom +adapters must preserve those bytes exactly and must not decompress or rewrite +them. + +Custom adapters can throw these root-exported error classes: - `DialCacheRedisPayloadError`; -- `DialCacheRedisPayloadEncodingError`; and -- `DialCacheRedisProtocolError`. +- `DialCacheRedisPayloadEncodingError`; +- `DialCacheRedisProtocolError`; and +- `DialCacheRedisPlaceholderLostError`. -They distinguish malformed payloads, unsupported encodings, and invalid Lua -reply domains in logs. DialCache records bounded `cache_read`, -`cache_read_timeout`, `cache_write`, or `invalidation` metrics by failure site. +They distinguish invalid runtime payload or reply shapes, unsupported +encodings, and lost tracked placeholders. DialCache records bounded +`cache_read`, `cache_read_timeout`, `cache_write`, or `invalidation` metrics by +failure site. -Shadow validation adds no Redis protocol operation. It composes the same -tracked read and write requests: `C0` and `C1` are tracked reads, and a clean -miss can use one tracked write. +Shadow validation adds no Redis protocol operation. It composes the ordinary +request shapes for the operation: `C0`, `C1`, and any clean-miss fill remain +tracked or untracked together. -#### Binary frame +### Binary frame Redis values use a compact binary frame: ```text byte 1 format version -bytes 2-9 Redis-created timestamp in milliseconds (uint64, big-endian) +bytes 2-9 creation timestamp or placeholder nonce (eight-byte region) byte 10 payload encoding (0 = UTF-8, 1 = raw binary) -bytes 11... serialized payload +bytes 11... opaque post-serialization payload ``` -Redis's Lua `struct` library packs and unpacks the timestamp. Redis TTL is -authoritative, so expiry metadata is not duplicated in the frame. - -The payload comes from the cache operation's serializer or `JsonSerializer` by -default. Custom serializers can return `string` or `Buffer`. Strings are -stored as UTF-8; Buffers are stored byte-for-byte without base64 expansion. -Adapters restore the same representation before calling `serializer.load`. +Version 1 is readable. A tracked placeholder uses version 0 and stores its +eight-byte nonce in the stamp region, so neither read path can serve it. The +stamp script promotes only its matching placeholder by replacing version and +nonce with version 1 and Redis time. + +An untracked frame is version 1 from the start and carries an informational +client-clock timestamp that untracked reads ignore. Redis TTL is authoritative, +so expiry metadata is not duplicated in the frame. + +The payload region contains the serializer output after any compression or raw +binary escaping. The frame encoding preserves whether that region is a string +or `Buffer`; strings use UTF-8 and Buffers need no base64 expansion. After the +adapter decodes the frame, DialCache interprets the optional compression +envelope and restores the serializer's representation before calling +`serializer.load`. diff --git a/docs/shadow-validation.md b/docs/shadow-validation.md index 27c13ae..302ee01 100644 --- a/docs/shadow-validation.md +++ b/docs/shadow-validation.md @@ -2,22 +2,23 @@ [Back to the README](../README.md) -Shadow validation lets a service exercise and inspect tracked Redis behavior -without letting the shadow path choose the caller's result. It is useful for -validating warm entries against the source of truth and for bootstrapping clean -misses before increasing the Redis serving ramp. +Shadow validation lets a service exercise and inspect Redis behavior without +letting the shadow path choose the caller's result. It is useful for validating +warm entries against the source of truth and for bootstrapping clean misses +before increasing the Redis serving ramp. Shadow mode is an operational rollout tool, not a new serving layer or a correctness boundary. It adds source and Redis work, provides best-effort -evidence through bounded metrics, and relies on the same invalidation, -serialization, deadline, and client-lifecycle contracts as the remote layer. +evidence through bounded metrics, and preserves each key's ordinary tracked or +untracked Redis mode. It relies on the same serialization, deadline, consistency, +and client-lifecycle contracts as the remote layer. ## At a glance | Path reached by the caller | Caller receives | Selected shadow work | | --- | --- | --- | -| Tracked, serving Redis hit | The decoded Redis value | Read the source later, compare it with the retained Redis payload, and confirm a candidate mismatch with one more tracked Redis read. | -| Valid remote policy, but the key is ramped out of Redis serving | The normal source result | Read Redis later without serving it. Compare a hit, or fill a clean miss from the caller-accepted source result. | +| Serving Redis hit, tracked or untracked | The decoded Redis value | Compare a later source read with the retained payload, then confirm a mismatch candidate with one same-mode Redis read. | +| Valid remote policy, but ramped out of Redis serving | The normal source result | Read Redis later without serving it. Compare a hit, or fill a clean miss from the accepted source result. | | Serving Redis miss | The normal source result | None. The ordinary request path already performs fallback and fill. | | Request-local or process-local hit | The in-memory value | None. Normal traversal never reached Redis. | @@ -27,8 +28,9 @@ served, selected for both, or selected for neither. ## Configure a shadow cohort -Shadow validation requires a tracked operation, a valid remote TTL, a metrics -adapter with the optional shadow hook, and a positive `shadow.ramp`: +Shadow validation requires a valid remote TTL, a metrics adapter with the +optional shadow hook, and a positive `shadow.ramp`. Invalidation tracking is +optional and selects the Redis consistency mode rather than shadow eligibility: ```ts import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache"; @@ -47,6 +49,7 @@ const getUser = dialcache.cached( keyType: "user_id", useCase: "GetUser", cacheKey: (userId) => userId, + // Optional for shadowing; adds watermark fencing to Redis reads and fills. trackForInvalidation: true, shadowComparator: (cached, source) => cached.id === source.id && cached.version === source.version, @@ -91,7 +94,6 @@ DialCache schedules shadow work only when all of these conditions hold: - the call began inside an enabled DialCache scope; - a Redis or Valkey adapter is configured; - normal traversal reaches the remote layer; -- the operation sets `trackForInvalidation: true`; - the resolved remote policy has a valid TTL; - the effective `shadow.ramp` is positive and selects the exact key; - the configured metrics adapter implements `shadowValidation`; and @@ -100,10 +102,9 @@ DialCache schedules shadow work only when all of these conditions hold: A remote serving ramp of `0` is eligible because it preserves a valid remote policy while excluding the key from serving. -Missing or invalid remote policy, provider failure, an untracked key, an -omitted metrics hook, an invalid or zero shadow ramp, cohort exclusion, an -earlier in-memory hit, or a disabled call does not start a shadow-only Redis -path. +Missing or invalid remote policy, provider failure, an omitted metrics hook, an +invalid or zero shadow ramp, cohort exclusion, an earlier in-memory hit, or a +disabled call does not start a shadow-only Redis path. An invalid runtime `shadow.ramp` does not disturb an otherwise valid caller-serving Redis hit. DialCache skips shadow work and records a @@ -121,6 +122,13 @@ DialCache validates this diagnostic leaf only after the metrics-hook, exact-key-cohort, and capacity gates; ineligible, cohort-excluded, and explicitly dropped work does not report that configuration error. +### Upgrade note for untracked keys + +Starting in `v0.15.0`, otherwise eligible untracked keys participate in shadow +work. A use case that already had a positive effective `shadow.ramp` can +therefore add source reads, Redis reads and fills, metrics, and opted-in logs. +Set its shadow ramp to `0` before upgrading if that work is not wanted. + See [Configuration and cache layers](configuration.md) for runtime-overlay precedence and policy validation. @@ -128,9 +136,9 @@ precedence and policy validation. ### Serving Redis hit -The request path performs its normal tracked Redis read and deserialization. -It returns that cached value without waiting for shadow work and retains the -exact serialized payload as `C0`. +The request path performs its normal Redis read and deserialization in the +key's tracked or untracked mode. It returns that cached value without waiting +for shadow work and retains the exact serialized payload as `C0`. On a later unreferenced event-loop turn, the shadow job: @@ -139,9 +147,11 @@ On a later unreferenced event-loop turn, the shadow job: 3. compares that snapshot with the source value `S`; and 4. performs a confirmation read only when the values differ. -The additional source call must be safe to run for observation. Process -coalescing means one serving Redis leader schedules at most one job for its -coalesced followers. +The additional source call must be safe to run for observation. With default +coalescing, one serving Redis leader schedules at most one job for its +followers. With `coalesce: false`, each caller can attempt scheduling; exact-key +shadow deduplication admits at most one concurrent job and reports the others +as `dropped`. ### Ramped down from Redis serving @@ -150,9 +160,9 @@ ramp is down, the caller runs and awaits the normal source loader. Shadow work reuses that same caller-owned promise as `S`; it does not invoke the loader a second time. -The detached job reads tracked Redis as `C0`. A hit is compared with `S`. A -clean miss can be filled from `S` using the invocation's resolved remote TTL -snapshot. +The detached job reads Redis as `C0` in the key's existing mode. A hit is +compared with `S`. A clean miss can be filled from `S` in that same mode using +the invocation's resolved remote TTL snapshot. The shadow Redis value never supplies the caller or populates request-local or process-local memory. If those in-memory layers are active, only the caller's @@ -162,20 +172,22 @@ When request-local and process-local caching are off and Redis serving is ramped down, caller invocations remain uncached and do not gain process coalescing merely because shadowing is enabled. Concurrent same-key callers can each run the source; shadow deduplication independently admits one shadow -job and reports the others as `dropped`. +job and reports the others as `dropped`. If an in-memory serving layer is +active, `coalesce: false` likewise keeps each caller's cache path independent. ## The `C0` / `S` / `C1` algorithm -`C0` is the original tracked Redis payload: either the payload that served the -caller or the result of the detached ramped-down read. `S` is the successfully -accepted source value. `C1` is an optional tracked confirmation read. +`C0` is the original Redis payload: either the payload that served the caller or +the result of the detached ramped-down read. `S` is the successfully accepted +source value. `C1` is an optional confirmation read in the same tracked or +untracked mode as `C0`. 1. Obtain `C0`. 2. If `C0` is `null`, follow the clean-miss fill path described below. 3. Otherwise, obtain `S`, deserialize a new snapshot from `C0`, and compare the cached and source values. 4. When they match, emit `match`; no confirmation read is needed. -5. When they differ, read tracked Redis again as `C1`, bypassing the +5. When they differ, read Redis again as `C1` in the same mode, bypassing the request-local and process-local caches. 6. If `C1` is missing or its bytes differ from `C0`, emit `superseded`. 7. If `C1` is byte-identical to `C0`, emit `mismatch`. @@ -186,32 +198,39 @@ confirmation. DialCache does not deserialize `C1`, compare it with `S`, or chase another version. `mismatch` therefore means that the exact observed Redis payload survived one -tracked confirmation read after a semantic disagreement. It is not a -cross-system atomic snapshot or a guarantee that the mismatch still exists. -`superseded` means only that the original observation could not be confirmed. +confirmation read after a semantic disagreement. It is not a cross-system +atomic snapshot or a guarantee that the mismatch still exists. For an +untracked key, it is also not proof of primary freshness or invalidation +safety. `superseded` means only that the original observation could not be +confirmed. No non-null `C0` is repaired, overwritten, invalidated, or given a refreshed TTL. That rule also applies when detached deserialization fails. ### Clean-miss fill -A clean miss means the tracked semantic Redis read returned `null`. It does -not include a non-null payload that the serializer cannot load. +A clean miss means the semantic Redis read returned `null`. It does not include +a non-null payload that the serializer cannot load. On the ramped-down path, DialCache can serialize the caller-accepted `S` and -attempt one ordinary tracked Redis write with the resolved TTL: +attempt one ordinary Redis write in the key's existing mode with the resolved +TTL: - `filled` means the client returned `true` before the shadow deadline; -- `fill_blocked` means the invalidation watermark returned `false`; and -- `fill_error` means serialization or the Redis write failed. +- `fill_blocked` means a tracked invalidation watermark returned `false`; and +- `fill_error` means preparing the payload (serialization or compression) or + writing it to Redis failed. A source rejection or caller fallback timeout never produces an accepted `S` and never starts the fill. Once serialization has finished, DialCache checks the whole-job deadline again before dispatching the write. -The `C0` read and fill are not atomic. The fill is a normal tracked overwrite, -not a compare-and-set or write-if-still-missing operation. Another writer can +The `C0` read and fill are not atomic. The fill is a normal overwrite, not a +compare-and-set or write-if-still-missing operation. Another writer can populate Redis after `C0` misses and then be overwritten by the shadow fill. +Tracked writes retain their watermark fence. An untracked fill has no fence and +uses ordinary TTL-based last-writer-wins publication, so an older accepted +source value can overwrite a concurrent newer value and remain until expiry. ## Comparison semantics @@ -374,23 +393,33 @@ off. See [Redis and Valkey](redis.md) for the complete custom-client, payload, deadline, and connection-lifecycle contracts. -## Invalidation and race boundaries - -Shadow mode is limited to invalidation-tracked keys. Both `C0` and `C1` use the -tracked read protocol, which atomically checks the value timestamp against the -watermark. Bundled adapters route tracked reads to the primary. - -A clean-miss fill uses the same serializer, Redis-time timestamp, value TTL, -and watermark-aware tracked write as an ordinary fill. A future watermark can -reject it as `fill_blocked`. Size `futureBufferMs` to cover the complete source, -serialization, client queue, network, script, and write interval if stale -publication protection matters. - -The watermark fences the tracked Redis write; it does not make the earlier -`C0` read and later fill atomic. It also does not synchronously invalidate -request-local or process-local entries. Shadow mode never evicts those layers. - -See [Targeted invalidation](invalidation.md) for the clock, durability, +## Consistency modes and race boundaries + +Shadowing preserves the operation's existing Redis mode: + +- **Tracked keys:** `C0` and `C1` use the watermark-aware read protocol, which + atomically checks the value timestamp against the invalidation watermark. + Bundled cluster adapters explicitly route these reads to the primary; a + standalone node-redis client must already target the authoritative endpoint. + A clean-miss fill uses the ordinary tracked write and can be rejected as + `fill_blocked` by a future watermark. +- **Untracked keys:** `C0` and `C1` use the ordinary one-key read route without a + watermark or shadow-specific primary guarantee. A clean-miss fill uses the + ordinary TTL write. Compliant untracked writes do not produce + `fill_blocked`. + +Both modes use the operation's serializer and value TTL. Tracked publication +receives a Redis-time timestamp from the stamp script; an untracked write uses +an informational client-clock timestamp that untracked reads never consult. + +Neither mode makes the initial `C0` read and later fill atomic. For tracked +keys, size `futureBufferMs` to cover the complete source, serialization, client +queue, network, and write interval when stale-publication protection matters. +The watermark fences only the tracked Redis write; it does not synchronously +invalidate request-local or process-local entries. Shadow mode never evicts +those layers. + +See [Targeted invalidation](invalidation.md) for the tracked clock, durability, retention, and future-buffer contracts. ### Command amplification @@ -398,10 +427,10 @@ retention, and future-buffer contracts. | Selected path | Added source work | Added Redis work | | --- | --- | --- | | Serving Redis hit, semantic match | One observational source read | None beyond the serving read | -| Serving Redis hit, mismatch candidate | One observational source read | One tracked confirmation read | -| Ramped-down Redis hit, semantic match | None beyond the caller's source read | One tracked `C0` read | -| Ramped-down Redis hit, mismatch candidate | None beyond the caller's source read | Tracked `C0` and `C1` reads | -| Ramped-down clean Redis miss | None beyond the caller's source read | One tracked `C0` read and at most one tracked write | +| Serving Redis hit, mismatch candidate | One observational source read | One same-mode confirmation read | +| Ramped-down Redis hit, semantic match | None beyond the caller's source read | One same-mode `C0` read | +| Ramped-down Redis hit, mismatch candidate | None beyond the caller's source read | Same-mode `C0` and `C1` reads | +| Ramped-down clean Redis miss | None beyond the caller's source read | One same-mode `C0` read and at most one write | | Serving Redis miss | None beyond the ordinary path | None beyond the ordinary read and fill | Capacity limits bound concurrent jobs, not total work over time. Measure source @@ -418,8 +447,8 @@ capacity cap, reports one bounded terminal outcome: | `mismatch` | They differed and byte-identical `C1` confirmed the original `C0`. | | `superseded` | They differed, but `C1` was missing or had different bytes. | | `filled` | A clean miss was populated successfully before the deadline. | -| `fill_blocked` | The invalidation watermark rejected the clean-miss fill. | -| `fill_error` | Serialization or the clean-miss Redis write failed. | +| `fill_blocked` | A tracked invalidation watermark rejected the clean-miss fill. | +| `fill_error` | Preparing the payload (serialization or compression) or writing the clean-miss fill failed. | | `redis_error` | The initial detached `C0` read failed or reached its read deadline. | | `source_error` | The source loader rejected without being the caller's own DialCache fallback timeout. | | `deserialization_error` | The retained non-null `C0` could not be deserialized. | @@ -459,7 +488,9 @@ Before increasing `shadow.ramp`: - confirm the source loader is safe to invoke observationally on serving hits; - use a valid remote TTL with the serving ramp at `0`; -- enable tracked invalidation and choose a defensible `futureBufferMs`; +- choose the Redis consistency mode deliberately: for tracked keys, configure a + defensible `futureBufferMs`; for untracked keys, accept TTL-based + last-writer-wins fills without invalidation or a primary-read guarantee; - verify serializer, comparator, Redis client, source, and telemetry budgets; - start with a small per-instance capacity and measure `dropped`; - account for the command amplification above; and @@ -493,8 +524,7 @@ During shutdown: 3. await request-path cache calls and invalidations; 4. use source, Redis-client, serializer, and telemetry-native controls to drain or terminate their work; -5. dispose adapter-owned resources; and -6. close underlying connections. +5. close underlying connections after their remaining work drains. Unreferenced shadow scheduling means the process may exit before an outcome is delivered. Already-started source reads, serializers, Redis commands, or