From fb983e292ebfc9460eb22175302620a3a44d362d Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 19 May 2026 10:05:42 -0400 Subject: [PATCH 1/2] build(pnpm): migrate to pnpm 11 + add minimumReleaseAge + pin Node 22 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #160. Three TS workspaces (clients/ts, tests/e2e/sdk, docs) and the setup-env composite action all jump from pnpm 10.33 to 11.1.3. pnpm 11 deprecates the `pnpm` field in package.json — `onlyBuiltDependencies` moved into a `pnpm-workspace.yaml` next to each package.json, renamed to `allowBuilds:` with the new map-of-booleans syntax. Same files hold a 7-day `minimumReleaseAge` (10080 minutes), so pnpm refuses freshly published packages until npm/jsr have had a week to flag a compromise. Node 22 is pinned via a new repo-root `.nvmrc` to match what CI's `setup-node` already uses. The issue flagged Node 26 crashing Vitest with a V8 abort; that's no longer reproducible after the recent vitest@^4 bump in clients/ts, but the pin still avoids Node-major surprises and gives `nvm`/`fnm`/`volta` users a one-shot version pick. Note on `electron-winstaller`: the issue's open question about whether it was a hidden transitive — it isn't. The warning came from a stray `/Users/eric/pnpm-workspace.yaml` in the local dev's home directory (pnpm 11 walks up looking for a workspace root). Not a WaveHouse dep, correctly omitted from `allowBuilds`. Verified: - make build-sdk green - make build-docs green - make test-sdk 120/120 passed (Node 26 + pnpm 11 — no crash) - pnpm install --frozen-lockfile clean in all three workspaces - Pre-existing typecheck error in clients/ts/src/cli/codegen.ts (missing @types/node) is unrelated and reproduces on main. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/actions/setup-env/action.yml | 2 +- .nvmrc | 1 + AGENTS.md | 7 ++++--- README.md | 2 +- clients/ts/package.json | 9 ++------- clients/ts/pnpm-workspace.yaml | 21 +++++++++++++++++++++ docs/package.json | 9 +-------- docs/pnpm-workspace.yaml | 12 ++++++++++++ docs/src/content/docs/development.md | 8 ++++---- tests/e2e/sdk/package.json | 7 +------ tests/e2e/sdk/pnpm-workspace.yaml | 7 +++++++ 11 files changed, 55 insertions(+), 30 deletions(-) create mode 100644 .nvmrc create mode 100644 clients/ts/pnpm-workspace.yaml create mode 100644 docs/pnpm-workspace.yaml create mode 100644 tests/e2e/sdk/pnpm-workspace.yaml diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 5a77a546..7141e18c 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -68,7 +68,7 @@ runs: # ask pnpm for the actual store path. - uses: pnpm/action-setup@0e279bb959325dab635dd2c09392533439d90093 # v6.0.8 with: - version: "10.33.0" + version: "11.1.3" run_install: false - name: Resolve pnpm store directory diff --git a/.nvmrc b/.nvmrc new file mode 100644 index 00000000..2bd5a0a9 --- /dev/null +++ b/.nvmrc @@ -0,0 +1 @@ +22 diff --git a/AGENTS.md b/AGENTS.md index 3232dfb2..1a6cb95e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -129,9 +129,10 @@ Tooling notes: auto-installed to `.bin/_/` on first `make lint` (or via `make tools`). Not in `go.mod` — its dependency tree conflicts with the main module. -- `pnpm` (>= 10.33) and `Node.js` (>= 20) must be on your PATH; the SDK and - E2E test harnesses both shell out to `pnpm`. `make tools` runs - `pnpm install --frozen-lockfile` in `clients/ts/` and `tests/e2e/sdk/`. +- `pnpm` (>= 11.1) and `Node.js` (22 LTS — pinned via `.nvmrc` at the repo + root, matches CI) must be on your PATH; the SDK and E2E test harnesses + both shell out to `pnpm`. `make tools` runs `pnpm install + --frozen-lockfile` in `clients/ts/` and `tests/e2e/sdk/`. - `GNU Make 4+` is required (uses `--output-sync=target`); macOS ships BSD Make 3.81 which will not parse the Makefile. See `docs/development.md` § Prerequisites for the full setup checklist. diff --git a/README.md b/README.md index a437c934..3715797e 100644 --- a/README.md +++ b/README.md @@ -116,7 +116,7 @@ WaveHouse is built as an application, not a library — `internal/` packages are ## 💻 Local Development -You'll need **Go 1.26+, GNU Make 4+, Docker (or Podman) with Compose v2, Node.js 20+, and pnpm 10+** on your PATH — see [docs/development.md § Prerequisites](docs/development.md#prerequisites) for the full list, version requirements, and macOS gotchas (BSD Make 3.81 won't work). +You'll need **Go 1.26+, GNU Make 4+, Docker (or Podman) with Compose v2, Node.js 22 LTS (pinned via `.nvmrc`, matches CI), and pnpm 11+** on your PATH — see [docs/development.md § Prerequisites](docs/development.md#prerequisites) for the full list, version requirements, and macOS gotchas (BSD Make 3.81 won't work). For building and testing WaveHouse locally with hot-reload: diff --git a/clients/ts/package.json b/clients/ts/package.json index c381253d..f0eaf439 100644 --- a/clients/ts/package.json +++ b/clients/ts/package.json @@ -2,7 +2,7 @@ "name": "@wavehouse/sdk", "version": "0.1.0", "description": "TypeScript SDK for WaveHouse — schema-aware real-time API gateway for ClickHouse", - "packageManager": "pnpm@10.33.0", + "packageManager": "pnpm@11.1.3", "type": "module", "main": "./dist/index.cjs", "module": "./dist/index.js", @@ -48,10 +48,5 @@ "sse", "query-builder", "typescript" - ], - "pnpm": { - "onlyBuiltDependencies": [ - "esbuild" - ] - } + ] } diff --git a/clients/ts/pnpm-workspace.yaml b/clients/ts/pnpm-workspace.yaml new file mode 100644 index 00000000..cab94d82 --- /dev/null +++ b/clients/ts/pnpm-workspace.yaml @@ -0,0 +1,21 @@ +# pnpm 11+ moved settings out of package.json#pnpm into this file. +# This is a single-package "workspace" — only used for settings, no +# `packages:` field. (Each WaveHouse pnpm project ships its own copy +# of this file; they're not one shared monorepo.) + +# Build scripts pnpm is allowed to run during install. Without this, +# pnpm 11 errors out with ERR_PNPM_IGNORED_BUILDS on any postinstall +# script for an unlisted package. +# esbuild — postinstall fetches the platform-native binary; tsup + +# vitest both depend on it. +# To add a new entry: `pnpm approve-builds ` (interactive) or +# edit this file directly with `: true`. +allowBuilds: + esbuild: true + +# Refuse packages published in the last 7 days (10080 minutes) so npm +# has time to surface compromised releases before they land in our +# lockfile. Use `minimumReleaseAgeExclude: [pkg-name]` for one-off +# overrides if a hotfix release needs to land before the window +# expires. +minimumReleaseAge: 10080 diff --git a/docs/package.json b/docs/package.json index 42e0921c..98fba169 100644 --- a/docs/package.json +++ b/docs/package.json @@ -3,7 +3,7 @@ "type": "module", "version": "0.0.1", "private": true, - "packageManager": "pnpm@10.33.0", + "packageManager": "pnpm@11.1.3", "scripts": { "dev": "astro dev", "start": "astro dev", @@ -33,12 +33,5 @@ }, "engines": { "node": ">=20.0" - }, - "pnpm": { - "onlyBuiltDependencies": [ - "esbuild", - "sharp", - "workerd" - ] } } diff --git a/docs/pnpm-workspace.yaml b/docs/pnpm-workspace.yaml new file mode 100644 index 00000000..10653a7a --- /dev/null +++ b/docs/pnpm-workspace.yaml @@ -0,0 +1,12 @@ +# See clients/ts/pnpm-workspace.yaml for full rationale. Kept in sync +# across all three WaveHouse pnpm projects manually. +# +# Docs needs extra build scripts beyond esbuild: +# sharp — native image pipeline used by Astro's image optimizer. +# workerd — Cloudflare Workers runtime fetched by wrangler. +allowBuilds: + esbuild: true + sharp: true + workerd: true + +minimumReleaseAge: 10080 diff --git a/docs/src/content/docs/development.md b/docs/src/content/docs/development.md index 9e3d8c25..b0a4466e 100644 --- a/docs/src/content/docs/development.md +++ b/docs/src/content/docs/development.md @@ -17,8 +17,8 @@ You need these on your `PATH` before any `make` recipe will work end-to-end: | **GNU Make** | **4.0+** | The Makefile uses `--output-sync=target` (Make 4 only) and bash-pinned recipes. macOS ships with BSD Make 3.81, which **will not work** | macOS: `brew install make` then use `gmake` or put `$(brew --prefix make)/libexec/gnubin` on your PATH. Linux: usually already installed | | **bash** | 4+ recommended | Recipes are pinned to `bash`; the helper scripts under `scripts/` use `set -euo pipefail` and bash arrays | macOS default is bash 3.2 (works for current recipes, but `brew install bash` is safer); Linux distros ship 4+ | | **Docker** *(or Podman)* | Engine 20.10+ with the Compose **v2** plugin (`docker compose`, no hyphen) | Compose stacks under `deployments/compose/` and `tests/e2e/compose.yaml`; integration tests boot a ClickHouse testcontainer | [Docker Desktop](https://docs.docker.com/get-docker/), [colima](https://github.com/abiosoft/colima), or [Podman](https://podman.io) with `podman-compose` / the `podman compose` plugin. The testcontainers Go library also honors `DOCKER_HOST` for rootless Podman setups | -| **Node.js** | 20+ | Runtime for pnpm and the Vitest suites | [nodejs.org](https://nodejs.org/) or `nvm`/`fnm`/`volta` | -| **pnpm** | 10.33+ (pinned via `packageManager` in `clients/ts/package.json`, `tests/e2e/sdk/package.json`, and `docs/package.json`) | Package manager for the TypeScript SDK, E2E test harness, and docs site; `make build-sdk`, `make test-sdk`, `make test-e2e`, `make build-docs`, `make dev-docs`, `make preview-docs` all shell out to `pnpm` | `corepack enable && corepack prepare pnpm@10.33.0 --activate` (recommended), or `npm i -g pnpm` | +| **Node.js** | 22 LTS — pinned via `.nvmrc` at the repo root | Runtime for pnpm and the Vitest suites. Pinned to match CI (`setup-node` uses 22) and to avoid Node-major surprises; older Vitest versions in this repo were known to crash on Node 26 with a V8 heap-allocation abort | [nodejs.org](https://nodejs.org/) or `nvm use` / `fnm use` / `volta` (all read `.nvmrc`) | +| **pnpm** | 11.1+ (pinned via `packageManager` in `clients/ts/package.json`, `tests/e2e/sdk/package.json`, and `docs/package.json`) | Package manager for the TypeScript SDK, E2E test harness, and docs site; `make build-sdk`, `make test-sdk`, `make test-e2e`, `make build-docs`, `make dev-docs`, `make preview-docs` all shell out to `pnpm` | `corepack enable && corepack prepare pnpm@11.1.3 --activate` (recommended), or `npm i -g pnpm` | | **git** + **curl** | any recent | `git` for source + version metadata in builds; `curl` is used by the Makefile to fetch the pinned `golangci-lint` binary into `.bin/` | usually preinstalled | ### Auto-installed by `make tools` @@ -36,8 +36,8 @@ Run `make tools` once after cloning to populate everything that doesn't have to go version # go1.26+ make --version # GNU Make 4.x docker compose version -node --version # v20+ -pnpm --version # 10.33+ +node --version # v22.x (matches .nvmrc and CI) +pnpm --version # 11.1+ ``` If any of those are wrong/missing, the Makefile recipes will fail with confusing errors (e.g. `--output-sync` is unrecognized on Make 3.81; `pnpm: command not found` on `make test-sdk`). diff --git a/tests/e2e/sdk/package.json b/tests/e2e/sdk/package.json index a7dd3222..1d632714 100644 --- a/tests/e2e/sdk/package.json +++ b/tests/e2e/sdk/package.json @@ -1,7 +1,7 @@ { "private": true, "name": "wavehouse-e2e", - "packageManager": "pnpm@10.33.0", + "packageManager": "pnpm@11.1.3", "type": "module", "scripts": { "test": "vitest run", @@ -14,10 +14,5 @@ "eventsource": "^4.1.0", "typescript": "^5.5.0", "vitest": "^3.0.0" - }, - "pnpm": { - "onlyBuiltDependencies": [ - "esbuild" - ] } } diff --git a/tests/e2e/sdk/pnpm-workspace.yaml b/tests/e2e/sdk/pnpm-workspace.yaml new file mode 100644 index 00000000..cfb6ffc0 --- /dev/null +++ b/tests/e2e/sdk/pnpm-workspace.yaml @@ -0,0 +1,7 @@ +# See clients/ts/pnpm-workspace.yaml for full rationale. Kept in sync +# across all three WaveHouse pnpm projects manually. + +allowBuilds: + esbuild: true + +minimumReleaseAge: 10080 From 447eb03ca4267857a57b76d29a1748de7a84d7c1 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 19 May 2026 10:19:05 -0400 Subject: [PATCH 2/2] build(pnpm): address PR #165 review (engines.node, README pin, CHANGELOG) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three one-liners from the PR review round: * docs/package.json#engines.node: >=20.0 → >=22.0 so corepack/volta flag local-dev mismatches before CI does (Claude SHOULD). * README.md: "pnpm 11+" → "pnpm 11.1+" — consistent with AGENTS.md/development.md and the 11.1.3 CI pin (CodeRabbit minor). * CHANGELOG.md: add Unreleased entries under ### Changed (toolchain bump) and a new ### Security subsection (minimumReleaseAge) (Claude + Gemini doc-sync MUST/SHOULD). The Gemini "automate minimumReleaseAge drift detection" suggestion is deliberately declined — three sibling files, all surfaced in any PR diff that touches them; the cost of a Make/CI check exceeds the cost of a one-line `grep minimumReleaseAge */pnpm-workspace.yaml` during review. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 5 +++++ README.md | 2 +- docs/package.json | 2 +- 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f5ee87b6..e0aced6e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,6 +27,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - WebSocket internal dispatch carries the subscribing table name through the merged channel instead of re-unmarshalling each payload to extract `table_name`. Drops the per-message redundant unmarshal and — for non-`EventMessage` payloads (raw-JSON pass-through) — keeps the outbound envelope's `"table"` field labelled with the actual subscription instead of falling back to `""`. No behaviour change for the standard `EventMessage` path. - Doc-sync for the `?topic=` → `?table=` rename: `docs/src/content/docs/getting-started.md` quick-start now shows `?table=clicks` (the previous `?topic=ingest.clicks` example would 400 against the new contract), and `docs/src/content/docs/sdk.md` `SharedWSManager` section now describes per-table (not per-topic) subscriptions and notes that the legacy NATS-style wildcards are no longer accepted server-side. - **Boot-time schema discovery no longer crashes the binary** (`cmd/wavehouse/main.go`, `internal/api/health.go`, `internal/discovery/discovery.go`, `docs/src/content/docs/api.md`, `docs/src/content/docs/deployment.md`). Previously, any error from the initial `SchemaRegistry.Refresh` — connection-refused, missing database, transient network blip — caused `cmd/wavehouse` to call `os.Exit(1)`. The supervisor would restart the process every ~10s in an unbounded loop, port 8080 never bound, and operators got `connection refused` on probes even when ClickHouse was otherwise healthy. Now the first Refresh failure is non-fatal: a new `api.BootState` is set with the diagnostic, the server still binds `:8080`, and a background goroutine calls the new `SchemaRegistry.RetryRefresh(ctx, 2s, 60s, onAttempt)` with exponential backoff until success or shutdown. While `BootState.Err()` is non-nil, `/health` returns 503 with `{"status":"degraded","error":"…"}` so an operator can `curl /health` to learn why the gateway isn't ready to serve traffic instead of grepping a restart-loop log; `/ready` returns 503 with `{"status":"not ready",…}` for the same reason. Once `BootState.Set(nil)` fires (either from the initial sync `Refresh` or from `SchemaRegistry.RetryRefresh` success), `/health` stays at `200 OK` for the rest of the process lifetime — that flip is sticky and reflects "boot completed once." `/ready`, by contrast, remains conditional on current ClickHouse reachability: it can flip back to `503` on transient runtime blips after boot, since readiness is a probe of "can I serve traffic right now," not "did I ever boot." Resolves #95. +- **Frontend toolchain pinned: pnpm 10.33 → 11.1.3, Node 22 LTS via `.nvmrc`** (`.nvmrc`, `.github/actions/setup-env/action.yml`, `clients/ts/package.json`, `clients/ts/pnpm-workspace.yaml`, `docs/package.json`, `docs/pnpm-workspace.yaml`, `tests/e2e/sdk/package.json`, `tests/e2e/sdk/pnpm-workspace.yaml`, `AGENTS.md`, `README.md`, `docs/src/content/docs/development.md`). Closes #160. Three TS workspaces (`clients/ts`, `tests/e2e/sdk`, `docs`) plus the `setup-env` composite action bump from `pnpm@10.33.0` to `pnpm@11.1.3`. pnpm 11 deprecated the `pnpm` field in `package.json` — `onlyBuiltDependencies` moved into a new per-workspace `pnpm-workspace.yaml` and was renamed to `allowBuilds:` with a map-of-booleans syntax (each workspace ships its own file even though they're not a single monorepo — having them in-tree shadows any stray ancestor `pnpm-workspace.yaml`, which is how pnpm 11 walks the directory tree). `clients/ts` and `tests/e2e/sdk` allow `esbuild`; `docs` additionally allows `sharp` (Astro image pipeline) and `workerd` (Cloudflare Workers runtime). New repo-root `.nvmrc` pins Node 22 LTS — matches the `setup-env` action's existing `node-version: "22"` — and `docs/package.json#engines.node` bumps from `>=20.0` to `>=22.0` so corepack / volta warn local devs before CI does. The original issue's open question about `electron-winstaller` resolved to a contamination from a stray `pnpm-workspace.yaml` in the developer's home directory, not a real WaveHouse transitive — correctly omitted from `allowBuilds`. The Node 26 Vitest crash mentioned in the issue is no longer reproducible after the recent `vitest@^4` bump in `clients/ts` (#103); the `.nvmrc` still lands because pinning has independent value (CI parity + Node-major-surprise guard). + +### Security + +- **`minimumReleaseAge: 10080` (7 days) on every pnpm workspace** (`clients/ts/pnpm-workspace.yaml`, `docs/pnpm-workspace.yaml`, `tests/e2e/sdk/pnpm-workspace.yaml`): pnpm 11 will refuse to install any package published in the last seven days, giving npm and security researchers time to flag a compromised release before it lands in our lockfile. Existing locked versions are grandfathered. For a one-off override on an urgent hotfix release, list the package under `minimumReleaseAgeExclude:` in the same file. Part of #160. ### Fixed - **Ingest worker no longer infinite-retries permanent delete errors** (`internal/ingest/bento.go`, `internal/ingest/bento_test.go`, `docs/src/content/docs/architecture.md`, `AGENTS.md`): `jsInput.Read`'s `action: "delete"` block called `m.Nak()` on every `chConn.Exec` failure, which JetStream interprets as "redeliver immediately." A delete whose error was *deterministic* (syntax error, unknown table, malformed identifier) would loop forever — clogging the buffer consumer, burning CPU, and spamming logs with the same message. Phase 1 of issue #91: every delete-Exec error is now treated as permanent. The original NATS envelope is published to `dlq.` (reusing the existing `bentoDLQDropped` counter when the DLQ publish itself fails) and the message is `DoubleAck`'d so it leaves the main queue. Issue #91 stays open after this lands as the Phase 2 tracker for transient-vs-permanent error classification (timeouts and network errors should still `Nak()` for retry); Phase 1 alone is the stopgap, Phase 2 makes the trade-off acceptable in production. diff --git a/README.md b/README.md index 88ee612f..98fe76e8 100644 --- a/README.md +++ b/README.md @@ -116,7 +116,7 @@ WaveHouse is built as an application, not a library — `internal/` packages are ## 💻 Local Development -You'll need **Go 1.26+, GNU Make 4+, Docker (or Podman) with Compose v2, Node.js 22 LTS (pinned via `.nvmrc`, matches CI), and pnpm 11+** on your PATH — see [docs/development.md § Prerequisites](docs/development.md#prerequisites) for the full list, version requirements, and macOS gotchas (BSD Make 3.81 won't work). +You'll need **Go 1.26+, GNU Make 4+, Docker (or Podman) with Compose v2, Node.js 22 LTS (pinned via `.nvmrc`, matches CI), and pnpm 11.1+** on your PATH — see [docs/development.md § Prerequisites](docs/development.md#prerequisites) for the full list, version requirements, and macOS gotchas (BSD Make 3.81 won't work). For building and testing WaveHouse locally with hot-reload: diff --git a/docs/package.json b/docs/package.json index 98fba169..3a1ebdb7 100644 --- a/docs/package.json +++ b/docs/package.json @@ -32,6 +32,6 @@ "wrangler": "^4.81.0" }, "engines": { - "node": ">=20.0" + "node": ">=22.0" } }