One Batcher trait, shaped like the one half_join had to write - #844
Merged
Conversation
`dogsdogsdogs::half_join` declared its own `Batcher` trait, because
`trace::Batcher` did not fit a consumer that stages updates in a form of
its own and releases them by a total order rather than an antichain. Two
traits with the same name and the same job, one of them half the size.
This adopts the smaller one, and deletes the duplicate.
pub trait Batcher<T, C0, C1> {
fn insert(&mut self, container: &mut C0);
fn extract<'a>(&'a mut self, upper: AntichainRef<'_, T>)
-> (Option<C1>, AntichainRef<'a, T>);
}
Four consequences, in rough order of how much they matter.
Input and output types are now distinct. `trace::Batcher` pinned them to
one associated type, which is why `arrange_core` carried a chunker whose
container had to equal `Ba::Output`. A batcher may now release whatever
its consumer means by a batch: half_join's is a sequence of chunks, and
arrange's is a chain it still hands to a `Builder`, which a following
commit narrows to the batch itself.
Construction leaves the trait. It has to: a trait meant to cover both a
merge batcher and, later, a builder-shaped one cannot mandate a
`new(logger, operator_id)` constructor, since only the arrange path has
a differential logger or an operator id to give. `arrange_core` takes the
constructor as an argument and `MergeBatcher::new` is inherent. Call
sites move the batcher from turbofish to argument position and are
otherwise unchanged.
The `Description` leaves too. `MergeBatcher::lower` existed only to
manufacture one, duplicating `arrange_core`'s `prev_frontier`; the
operator now builds it from the frontiers it already tracks.
`seal` and `frontier` collapse into one call, so the ordering invariant
between them ("the frontier after the most recent seal") is gone. The
retained frontier is reported by the extraction that determines it.
`insert` takes its container by reference, as `Builder::push` does, and
the implementor decides whether to claim the allocation or drain it. The
recycling decision belongs there rather than at the call site, which no
longer writes `std::mem::take`; `arrange_core` drops a `Default` bound as
a result. This also keeps the door open for a builder-shaped batcher:
`reduce` allocates one buffer outside its per-key loop and hands it over
by reference every key, which a by-value `insert` would cost it.
One behaviour change worth naming: the empty-batch branch used to call
`seal` and discard the result, to advance the lower bound that no longer
exists. Dropping that call also drops the full chain merge it performed
on every empty frontier advance. No held capability precedes the input
frontier there, so no update does either, and nothing was extractable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
frankmcsherry
force-pushed
the
one-batcher-trait
branch
from
August 26, 2026 01:15
c5dda33 to
2b40d56
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dogsdogsdogs::half_joindeclared its ownBatchertrait, becausetrace::Batcherdid not fit a consumer that stages updates in a form of its own and releases them by a total order rather than an antichain. Two traits with the same name and the same job, one of them half the size. This adopts the smaller one and deletes the duplicate.Four consequences, in rough order of how much they matter.
Input and output types are now distinct.
trace::Batcherpinned them to one associated type, which is whyarrange_corecarried a chunker whose container had to equalBa::Output, and whyarrangeneeded a doc caveat telling you to callarrange_coredirectly when they differ. A batcher may now release whatever its consumer means by a batch. half_join's is a sequence of chunks; arrange's is a chain it still hands to aBuilder, which a following commit narrows to the batch itself.Construction leaves the trait. This is the piece that looks like churn and isn't. A trait meant to cover both a merge batcher and — later — a builder-shaped one cannot mandate a
new(logger, operator_id)constructor, because only the arrange path has a differential logger or a timely operator id to give.arrange_coretakes the constructor as an argument, andMergeBatcher::newbecomes inherent. Call sites move the batcher from turbofish to argument position:The
Descriptionleaves too.MergeBatcher::lowerexisted only to manufacture one, duplicatingarrange_core'sprev_frontier. The operator now builds the description from the frontiers it already tracks, and the field is gone.sealandfrontiercollapse into one call, so the invariant between them — "the frontier of elements remaining after the most recent call toseal" — no longer needs stating. The retained frontier is reported by the extraction that determines it.One behaviour change
The empty-batch branch used to call
sealand discard the result, to advance the lower bound that no longer exists. Dropping that call also drops the full chain merge it performed on every empty frontier advance. Correctness is unaffected: no held capability precedes the input frontier in that branch, so no update does either, and nothing was extractable. It is the one thing here a benchmark could notice, in either direction.+181 −194. Whole workspace builds warning-free and the test suite passes.
🤖 Generated with Claude Code