Feature(#1712): route variable-sized allocations through size classes (retire unpooled) - #1
Open
zeuchste wants to merge 7 commits into
Open
Conversation
The global buffer manager only handed out a single fixed buffer size; any other size fell through to the slower per-thread unpooled path (heap-allocated control block, RW-locked chunk maps, deferred chunk free). This generalizes the pooled path into segregated power-of-two size classes so variable-sized requests are served O(1) from a lock-free pool, the same segregated-free-list idea used by Umbra, Velox and ClickHouse. Core (nes-memory): - FixedSizeClassPool: the existing fixed pool (folly::MPMCQueue + placement-new Native control blocks) generalized to one instance per size class, multi-region, with optional elastic growth. - BufferManager owns one pool per class; getBuffer(size) rounds up to the smallest fitting class, promotes to larger classes when momentarily empty, and falls back to the unpooled path only for the rare > max-class tail. - Pluggable provisioning: TotalBudgetSplit / EagerPerClass / LazyElastic. - Backward compatible: with no SizeClassConfig there is exactly one class (the default operator buffer size) and behavior is byte-for-byte unchanged. Consumers rerouted off the unpooled path (Phase 1): MemoryUtils::getBuffer, Arena oversized allocations, NetworkBindings child buffers. Config: worker keys enable_buffer_size_classes (+ min/max/policy/budget), default off; min/max validated as powers of two (PowerOfTwoValidation) and min <= max at startup. Tests/bench: SizeClassBufferTests (rounding, all 3 policies, recycle-to-class, promotion, lazy growth, unpooled fallback, 8-thread leak-free stress, rapidcheck property); self-contained BufferManagerBenchmark (size-class ~ fixed pooled, ~2-5x faster than unpooled). Full ctest 818/818 green; 502/502 functional systests pass with size classes enabled. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Covers powers of two (incl. 2^63), and rejects zero, non-powers-of-two, empty/non-numeric input, trailing characters, hex, and uint64 overflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Make the sized getBuffer()/getBufferNoBlocking() pure virtual in AbstractBufferProvider (BufferManager is the only implementer); removes the never-executed default bodies. - Extract NodeEngineBuilder::makeSizeClassConfig() and unit test the worker-config -> SizeClassConfig translation (disabled, enabled, min>max throws). - Extend SizeClassBufferTests: non-blocking exhaustion, TotalBudgetSplit per class, LazyElastic past ceiling, deepCopyBuffer, and blocking-throw on exhaustion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mock The sized getBuffer()/getBufferNoBlocking() became pure virtual in AbstractBufferProvider; implement them in the PagedVectorTest DirtyBufferProvider decorator (delegating to the wrapped BufferManager and applying the dirty-fill). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ze classes PagedVector and ChainedHashMap (entry space, storage pages, var-sized space) allocate via a new getPagedBuffer() helper: when the provider serves size classes (servesSizeClasses()), pull a fitting pooled buffer (getBufferNoBlocking) and fall back to unpooled only on miss; when size classes are off, use the unpooled path unchanged so we never consume the default data pool for state. Preserves the clean-failure (optional) semantics (nebulastream#1702). Verified: 36 hashmap/paged-vector unit tests pass; join+aggregation correct under size classes; the per-class log shows operator state now served from 256/512/1024B (and larger) classes instead of unpooled. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 45ebc53)
…e classes (core sites) Replace direct getUnpooledBuffer(size) with the size-class getPagedBuffer() at the variable-sized child buffer path (TupleBufferRef::getNewBufferForVarSized), the aggregation + hash-join window-trigger handler buffers, and the slice-cache allocation. With size classes enabled these now draw from a fitting pooled class under the unified budget; with them off they fall back to unpooled (regression-safe). Unpooled remains only the bounded oversize fallback inside getBuffer. Verified: NINJA_EXIT=0; default agg/join/varsized pass (unchanged); varsized join passes with size classes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 48d395c)
…ize classes (HJ/NLJ/median) Route the remaining variable-sized production sites -- the PagedVector main buffer in HJBuild, both NLJ slice sides, and the median aggregation -- through getPagedBuffer(). Together with the core-sites commit, every variable-sized production allocation now flows through the size-class path (unpooled remains only the bounded oversize fallback inside getBuffer), under one budget. Regression-safe (size classes off = unpooled). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit d3b5364)
zeuchste
force-pushed
the
backport/1705-operator-state-pooling
branch
from
June 22, 2026 16:23
2f35dce to
9170ee3
Compare
zeuchste
force-pushed
the
backport/1712-retire-unpooled
branch
from
June 22, 2026 16:23
145ce06 to
60ffdb8
Compare
zeuchste
force-pushed
the
backport/1705-operator-state-pooling
branch
3 times, most recently
from
June 29, 2026 11:18
8498077 to
85b047d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of EPIC nebulastream#1714. Routes the remaining 8 variable-sized production allocation sites through size classes (
getPagedBuffer), unifying every variable-sized allocation under one bounded budget; unpooled remains only the bounded oversize fallback. Stacked on nebulastream#1705; retarget to nebulastream#1706/main once nebulastream#1705 merges.🤖 Generated with Claude Code