Batch child node lookups to avoid N+1 queries - #6212
Draft
VPS-thodax wants to merge 1 commit into
Draft
Conversation
Walking the page tree issued one query per node to load its children, so rendering a menu, resolving a path segment by segment or collecting descendants scaled with the number of nodes. Batch "children of node X" lookups into one query per scope and tick, and descend into all children of a tree level at once in getDescendants instead of one after another. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PrvQarKxE7b5obfJnS3kRJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
depends on #6211
Problem
Walking the page tree loads the children of one node per query. Rendering a three-level menu over 200 root nodes issues 612
PageTreeNodequeries;getDescendants()and resolving a path segment by segment scale the same way.Reason
getChildNodes(),getNodeByPath()andpageTreeRootNodeList()all go throughqueryNodes(), which runs a live query per call whenever the scope has not been preloaded, so every node's children cost a round trip.getDescendants()additionally awaits one child after another, so those queries cannot even overlap.Fix
Lookups of the form "children of node X" go through a
DataLoaderkeyed by scope and parent: all such lookups in the same tick collapse into one query per scope (parentId in (…), with a separate condition for root nodes, which cannot be expressed inside$in).getDescendants()descends into all children of a level at once, so one query per tree level replaces one per node.Like the node loader in #6211, this loader and its cache are per request. They live on the read API instance, and
PageTreeReadApiServiceis request-scoped, so a batch is never shared between requests and there is nothing to invalidate when the tree changes.Scopes that were preloaded still short-circuit before the loader, so resolvers calling
preloadNodes()are unaffected. Lookups that are not keyed by a parent keep the previous live query —getNodes()filters by category and document type across a whole scope and supports sorting and pagination, which cannot share a batch.Decisions
sort,limitandoffset. Callers passing different sorting or pagination cannot share a result set, so merging them into one batch would mean re-implementing pagination in memory.getNodes()is the only caller affected and stays on the live-query path.category,slugorexcludeHiddenInMenucondition in the query would split the batch again. Children of a parent are therefore fetched in full and filtered afterwards throughfilterPreloadedNodes()— the same in-memory filtering the preload path already does, so the results are unchanged while a few more rows are read.Verification
Measured on the demo API against a page tree of 200 root nodes, each four levels deep, counting statements in the PostgreSQL log. The query is
topMenu { id childNodes { id childNodes { id childNodes { id } } } }, chosen becausetopMenudeliberately does not preload:PageTreeNodequeriesThe response is byte-identical before and after, as are the ten page tree queries checked in #6211 — among them
pageTreeNodeList,mainMenu,topMenu,parentNodes,pageTreeFullTextSearch,paginatedPageTreeNodesandpaginatedRedirects.New unit tests cover the children of several nodes collapsing into one query, repeated lookups of the same node, nodes without children, root nodes batching together with child nodes, the remaining filters still being applied, one query per level in
getDescendants(), and one query per segment when several paths are resolved concurrently.Further information
Generated by Claude Code