Batch page tree node lookups to avoid N+1 queries

[VPS-thodax]

Problem

Resolving the content of a page runs one database query per internal link, plus one more per segment of each link's path. A page with 200 internal links pointing at nodes four levels deep issues 801 PageTreeNode queries:

select "p0".* from "PageTreeNode" as "p0" where "p0"."id" = '…' and "p0"."visibility" in ('Published') limit 1

The same node is queried repeatedly within a single request, and the query count grows with the number of links on the page.

Reason

PageTreeReadApi.getNode() writes a node into its cache only once the query has resolved. Blocks are transformed concurrently (Promise.all in transformToPlain), so every lookup issued before the first one returns misses the cache and runs its own query. nodePath() then walks up the tree with one getNode() per level, multiplying the effect.

Fix

Node lookups go through a DataLoader: loads happening in the same tick collapse into a single id in (…) query, and duplicate lookups of the same id are deduplicated while in flight. The number of queries is now bound by the depth of the page tree instead of the number of links — the page above drops from 801 queries to 5. getNodesByIds() uses the same loader, so it shares the batch and the cache.

The loader and its cache are per request. They live on the read API instance, and PageTreeReadApiService is request-scoped, so nothing is shared between requests — or between API instances. There is no cache to invalidate and no coherency to manage when running several replicas: a node published or unpublished elsewhere is visible on the very next request.

One behavior change within that request: lookups that find nothing are now cached too. That is what you want for a request; for a read API deliberately kept alive longer — createReadApi() in a console command, for example — a node that becomes visible mid-run is not picked up. The cache for found nodes already behaved this way.

Decisions

• Batches lookups by id only, not queryNodes(). getChildNodes(), getNodeByPath() and pageTreeRootNodeList() filter by slug, parent and scope instead of by id, so they need a different batching strategy. Left out to keep this change small. -> done in Batch child node lookups to avoid N+1 queries #6212

Verification

Measured on the demo API against a page whose internal links point at nodes four levels deep, counting statements in the PostgreSQL log. This is a local environment, so the ratios are the meaningful part, not the absolute timings:

Internal links	Queries before	Queries after	Time before	Time after
50	201	5	149 ms	45 ms
200	801	5	550 ms	89 ms

With 10 concurrent requests against a single API instance, throughput goes from 1.6 req/s to 10.6 req/s and median latency from 4.5 s to 0.84 s.

That the cache does not outlive the request was checked against the running API: renaming a node, and unpublishing and republishing one, each took effect on the very next request.

GraphQL responses are byte-identical before and after. Nine further page tree queries — among them pageTreeNodeList, mainMenu, topMenu, parentNodes, pageTreeFullTextSearch, paginatedPageTreeNodes and paginatedRedirects — also return identical responses.

New unit tests cover concurrent lookups of different ids collapsing into one query, repeated lookups of the same id, already-loaded nodes, nodes that do not exist, and resolving several paths with one query per tree level.

Further information

• Task: https://vivid-planet.atlassian.net/browse/COM-3137