diff --git a/.github/PUBLISHING.md b/.github/PUBLISHING.md deleted file mode 100644 index 6d173f1..0000000 --- a/.github/PUBLISHING.md +++ /dev/null @@ -1,56 +0,0 @@ -# Publishing - -Releases are driven by [changesets](https://github.com/changesets/changesets) and published to npm -with **Trusted Publishing**, so there is no long-lived npm token in this repository. - -## Day-to-day - -1. Make your change. -2. Run `npx changeset` and describe it. Pick `patch`, `minor` or `major`. -3. Commit the generated file in `.changeset/` alongside your change. - -When that lands on `main`, the Release workflow opens (or updates) a `chore: release` pull request -that bumps the version and writes `CHANGELOG.md`. Merging that PR publishes to npm. - -## One-time setup - -Trusted Publishing has to be enabled on the npm side; the workflow cannot do it for you. - -1. Publish `1.0.0` manually once, so the package exists — a trusted publisher is configured - per-package, so there is nothing to attach it to until then: - - ```bash - npm login - npm run build - npm publish - ``` - - `publishConfig.access` in `package.json` already marks it public, so no flag is needed. Do not add - `--provenance` here: provenance needs the OIDC token that only CI has, and the command will fail. - Every later release gets provenance automatically from the workflow. - -2. On , under **Trusted Publisher**, add a GitHub - Actions publisher: - - Organization or user: `lukapozega` - - Repository: `renderready` - - Workflow filename: `release.yml` - -3. Remove any `NPM_TOKEN` secret from the repository. It is no longer needed, and a leaked token is - the thing this setup exists to avoid. - -Every later release goes through the workflow. Publishing this way also attaches a provenance -attestation, so consumers can verify the tarball was built from this repository. - -## Requirements the workflow depends on - -- `id-token: write` permission, which is what mints the OIDC token. -- npm 11.5.1 or newer, which is why the workflow upgrades npm before publishing. -- The package must be public (`.changeset/config.json` sets `"access": "public"`). - -## Why `esbuild` is pinned - -`package.json` forces `esbuild` to `^0.28.1` through `overrides`. This looks like an unexplained pin -and is not: tsup depends on `^0.27.0`, and the 0.27 line carries -[GHSA-g7r4-m6w7-qqqr](https://github.com/advisories/GHSA-g7r4-m6w7-qqqr), so removing the override -brings the advisory straight back into `npm audit`. It is build-time only and never reaches -consumers. JSON cannot hold a comment, which is why the reason is recorded here. diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f163fb1..c19e7c4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -51,4 +51,4 @@ jobs: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # No NODE_AUTH_TOKEN and no NPM_TOKEN: publishing authenticates with # the OIDC token from `id-token: write`. This requires a one-time - # setup on npmjs.com — see .github/PUBLISHING.md. + # setup on npmjs.com diff --git a/README.md b/README.md index 19245c7..3097aae 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,8 @@ works on sites you do not control instead of only ones you have instrumented. - [Concurrency and capacity](#concurrency-and-capacity) - [Security](#security) - [Compatibility](#compatibility) + - [Migrating from `prerender/prerender`](docs/migrating-from-prerender.md) + - [Migrating from Rendertron](docs/migrating-from-rendertron.md) - [Requirements](#requirements) - [Contributing](#contributing) @@ -499,9 +501,6 @@ package can honestly promise. Use `allowedDomains` or a network policy. renderer exploit is not contained by the sandbox. If you render untrusted URLs, run the service in its own container with a seccomp profile and no network access beyond what it needs. -To report a vulnerability, please open a draft security advisory on GitHub rather than a public -issue. - ## Compatibility Deliberately **not** included, so you know what to expect: @@ -516,7 +515,11 @@ Deliberately **not** included, so you know what to expect: If you are moving from another prerendering service, the two things to check are that your application sets `window.renderReady` (or relies on network quiet, which needs no changes) and that -any soft-404 meta tags use the `renderready-` prefix documented above. +any soft-404 meta tags use the `renderready-` prefix documented above. Dedicated guides: + +- [Migrating from `prerender/prerender`](docs/migrating-from-prerender.md) — the repository now + returns a 404. +- [Migrating from Rendertron](docs/migrating-from-rendertron.md) — archived since October 2022. ## Requirements diff --git a/docs/migrating-from-prerender.md b/docs/migrating-from-prerender.md new file mode 100644 index 0000000..8508993 --- /dev/null +++ b/docs/migrating-from-prerender.md @@ -0,0 +1,144 @@ +# Migrating from `prerender/prerender` + +As of this writing, [`prerender/prerender`](https://github.com/prerender/prerender) returns a 404 — +the source repository is gone. The [npm package](https://www.npmjs.com/package/prerender) still +resolves and installs (last published 2024-09-12), so existing deployments and fresh installs both +keep working. What you lose is a path to a fix: no repository means no security patches, and no fix +for the one crash-recovery gap in its own design — if Chrome dies twice within a second of each +other, the old server calls `process.exit()` and depends on an external supervisor to bring it back, +with no retry loop of its own. Its browser recycling is otherwise reasonable: it restarts an idle +browser automatically, and restarts on a fixed schedule too if you use the documented `server.js` +entry point (or register the `browserForceRestart` plugin yourself on a custom build). + +The good news: renderready is behaviour-compatible with almost everything the old server did. Most +migrations are a URL change and an environment-variable rename, not an application rewrite. + +## The request + +The old server answered two shapes: `GET /` (the whole path _was_ the target URL — used by +`prerender-node` and friends) and `GET|POST /render?url=…` (the crawler API). renderready only +answers the second: + +```diff +- GET /http://localhost:8000/products/1 ++ GET /render?url=http%3A%2F%2Flocalhost%3A8000%2Fproducts%2F1 +``` + +If you're using a middleware package (`prerender-node`, `prerender_rails`, `Laravel-Prerender`), +check whether it can be pointed at `/render?url=` directly, or whether you need a thin proxy in +front that rewrites the catch-all form into a query parameter. The catch-all is gone on purpose — +it made every malformed request look like a render attempt. + +## The readiness flag + +Unchanged in behaviour, renamed: + +```diff +- window.prerenderReady = false; ++ window.renderReady = false; +``` + +Same semantics: define it as a boolean and it becomes authoritative — nothing is captured until it +turns `true`. Once it does, capture happens as soon as the network goes quiet or +`renderReadyDelay` (was `prerenderReadyDelay`) elapses, whichever comes first. If you never define +it, nothing changes: network-quiet still handles it. + +## Meta tags + +```diff +- ++ + +- ++ +``` + +Both are read from `` only and stripped from the output, same as before. + +## Headers + +The old server sent `X-Prerender: 1` to your origin so your app could detect the crawler. renderready +sends `X-RenderReady: 1` by default (configurable via `originHeaders`). If you have server-side logic +branching on that header, it needs the new name. + +Response-side: `x-prerender-504-reason` is now `x-renderready-error`, and `x-prerender-render-id` / +`x-prerender-render-at` (from the `addMetaTags` plugin) are `x-renderready-render-id` / +`x-renderready-render-at`, opt-in via `injectRenderMeta` rather than a plugin you register. + +## Environment variables + +Most names are unchanged. The ones that moved: + +| Old | New | +| ----------------------------------- | -------------------------- | +| `chromeLocation` (JS option)\* | `CHROME_PATH` | +| `RENDERING_ERROR_STATUS_CODE` | `RENDER_ERROR_STATUS_CODE` | +| `prerenderReadyDelay` (JS option)\* | `RENDER_READY_DELAY` | + +\* neither of these was ever an environment variable in the old server — only a JS constructor +option existed. Both gained an environment-variable form for the first time in renderready. + +`PORT`, `WAIT_AFTER_LAST_REQUEST`, `PAGE_DONE_CHECK_INTERVAL`, `PAGE_LOAD_TIMEOUT`, +`FOLLOW_REDIRECTS`, `TIMEOUT_STATUS_CODE`, `ALLOWED_DOMAINS`, `BASIC_AUTH_USERNAME`, +`BASIC_AUTH_PASSWORD` all mean exactly what they did. + +## Plugins → config flags and hooks + +The old server used a `server.use(plugin)` system with nine bundled plugins. renderready folds the +common ones into options — see the [README options tables](../README.md#options) for the full +list — and gives you [four hooks](../README.md#hooks) for the rest. + +| Plugin | renderready equivalent | +| ----------------------- | ------------------------------------------------------------------- | +| `whitelist`/`blacklist` | `allowedDomains` / `blockedDomains` | +| `basicAuth` | `basicAuth` option | +| `removeScriptTags` | `removeScriptTags` option, `true` by default (was opt-in) | +| `httpHeaders` | `metaStatusCode` option, `true` by default (was opt-in) | +| `addMetaTags` | `injectRenderMeta` option, `false` by default | +| `sendPrerenderHeader` | `originHeaders`, on by default | +| `blockResources` | `blockedResourceTypes` / `blockedUrlPatterns` | +| `browserForceRestart` | `recycleAfterMs` (age-based recycling is on by default, not opt-in) | + +The two behavioural defaults worth double-checking: script stripping and meta-directive handling +were things you had to register a plugin for; in renderready they're on unless you turn them off. + +Caching was never bundled in the old server either — the README listed an `s3-html-cache` plugin +as "coming soon," but it was never published. renderready doesn't ship a cache either; see +[Adding a cache](../README.md#adding-a-cache) if you want to add one. + +## What's gone + +- **`renderType=png|jpeg|pdf|har`.** HTML only. If you depended on the screenshot or PDF endpoints, + there's currently no replacement — Playwright makes them straightforward to add on top of + `createRenderer()` if you need to fork or wrap. +- **`_escaped_fragment_` query handling.** Google retired the AJAX crawling scheme in 2015; nothing + currently depends on it. +- **HAR file export.** + +## Chrome installation + +The old server expected Chrome to already be on the machine, at a hardcoded path per platform (and +`chromeLocation` to override it). renderready uses Playwright, which manages its own browser: + +```bash +npx playwright install chromium +``` + +`chromePath` / `CHROME_PATH` still let you point at a system Chrome instead. + +## Checklist + +- [ ] Point middleware or reverse-proxy rules at `/render?url=` instead of the catch-all path. +- [ ] Rename `window.prerenderReady` → `window.renderReady` in any app that sets it (skip if you + never set it — network-quiet is unaffected). +- [ ] Rename `prerender-status-code` / `prerender-header` meta tags. +- [ ] Update any code branching on the `X-Prerender` request header or `x-prerender-*` response + headers. +- [ ] Convert the `chromeLocation` constructor option (if you set one) to the `CHROME_PATH` env + var, and rename `RENDERING_ERROR_STATUS_CODE` → `RENDER_ERROR_STATUS_CODE` in your + environment. +- [ ] Run `npx playwright install chromium` instead of relying on a system Chrome install. +- [ ] If you used `blockResources` or `browserForceRestart`, read the corresponding option above — + behaviour is similar but the defaults differ. If you relied on caching, see + [Adding a cache](../README.md#adding-a-cache) — the old server's cache was never actually + bundled either. diff --git a/docs/migrating-from-rendertron.md b/docs/migrating-from-rendertron.md new file mode 100644 index 0000000..5f523a8 --- /dev/null +++ b/docs/migrating-from-rendertron.md @@ -0,0 +1,113 @@ +# Migrating from Rendertron + +[GoogleChrome/rendertron](https://github.com/GoogleChrome/rendertron) was archived in October 2022 +and hasn't had a commit since. This is a bigger migration than moving off `prerender/prerender` — +Rendertron's API shape is different in a few places — but the core behaviour you're relying on +carries over directly. + +## The request + +Rendertron embeds the target URL in the path. renderready uses a query parameter: + +```diff +- GET /render/https://example.com/products/1 ++ GET /render?url=https%3A%2F%2Fexample.com%2Fproducts%2F1 +``` + +This is the change every caller needs to make — a reverse-proxy rewrite rule, middleware +configuration, or whatever issues the render request has to switch from path-embedding to +query-encoding the URL. There's no way to make the query form accept a raw, unencoded URL in the +path; `?` and `&` inside the target URL would otherwise be ambiguous with the request's own query +string. + +`?mobile` did two things: it set mobile viewport dimensions and switched Rendertron's own +User-Agent to a mobile string. The dimensions map to a `width`/`height` override on the request +(or `viewportWidth`/`viewportHeight` if you run separately-configured instances for mobile and +desktop); the User-Agent swap maps to the `userAgent` override — set it explicitly if your app +does UA-sniffing to decide what markup or styles to serve, since changing the viewport size alone +won't trigger that logic. + +## Readiness — this is the good news + +Rendertron never had a page-side readiness signal. Its renderer "waits for the page load event and +for outstanding network requests to settle" — full stop. There's no equivalent of a custom flag +your app can set. + +renderready's network-quiet fallback is a direct behavioural match for this — it's what Rendertron +was already doing. **You don't have to change your application at all to get equivalent behaviour.** +`window.renderReady` is available if you _want_ apps to be able to cut a render short once they know +they're done, which Rendertron's approach never let you do, but adopting it is optional, not +required for migration. + +## Status codes + +Rendertron preserves the origin's status code, same as renderready. The soft-status meta tag +differs in both name and separator: + +```diff +- ++ +``` + +Rendertron has no documented equivalent of a header-injection meta tag (renderready's +`renderready-header`, for declaring things like a redirect `Location` from the page). If you were +working around that gap some other way, `renderready-header` may let you remove the workaround. + +## Caching + +Rendertron shipped three built-in cache backends — in-memory, filesystem, and Google Cloud +Datastore — with configurable TTL and entry limits. + +**renderready has no built-in cache**, deliberately: what to key on, how long to keep an entry, and +where to store it are decisions specific to your traffic, and an HTTP cache in front of the service +(a CDN, Varnish, nginx) is usually a better answer than anything baked into the renderer. See +[Adding a cache](../README.md#adding-a-cache) for the two hooks you need to build your own — +`onRequest` to check a cache before rendering, `onPageLoaded` to write to it after. + +If you were using Rendertron's `GET /invalidate/` to bust the cache, you'll need to replace it +with a purge call against whatever you put in front — a CDN purge API, or clearing your own cache +store directly. + +## What's gone + +- **Screenshots** (`GET /screenshot/`). Out of scope for v1 — Playwright makes this + straightforward to add on top of `createRenderer()` if you need it. +- **The `wc-inject-shadydom` query parameter.** Rendertron used this to force a ShadyDOM polyfill + for older Web Components v1 implementations. There's no equivalent flag in renderready; modern + Chromium (which is what Playwright drives) has broad native support for web components without + it in most cases. + +## Configuration + +Rendertron is configured through a `config.json` file, with only `PORT` and `HOST` overridable by +environment variable. renderready takes every option as either a constructor argument or an +environment variable — see the [README options tables](../README.md#options). There's no config +file format to translate; map your `config.json` values to the equivalent option names directly. + +The one Rendertron option with no renderready counterpart is `restrictedUrlPattern` (a regex +blocklist on the request path). The nearest equivalents are `blockedDomains` and +`blockedUrlPatterns`, though the latter matches against the target URL rather than the incoming +request path. + +## Browser installation + +Rendertron bundled Puppeteer, which downloads its own Chromium automatically on `npm install`. +renderready uses `playwright-core`, which doesn't — install the browser explicitly: + +```bash +npx playwright install chromium +``` + +This keeps the package install small; it costs you one extra command. + +## Checklist + +- [ ] Change every caller from `GET /render/` to `GET /render?url=`. +- [ ] If you relied on `?mobile`, configure `viewportWidth`/`viewportHeight` or pass per-request + `width`/`height` overrides instead. +- [ ] Rename any `render:status_code` meta tags to `renderready-status-code`. +- [ ] Replace Rendertron's built-in cache with an HTTP cache in front of the service, or wire up + `onRequest`/`onPageLoaded` — see [Adding a cache](../README.md#adding-a-cache). +- [ ] Replace any use of `GET /invalidate/` with a purge against whatever now sits in front. +- [ ] Run `npx playwright install chromium` — the browser is no longer bundled automatically. +- [ ] Drop `wc-inject-shadydom` from any request URLs; there's no equivalent flag.