Skip to content

One Batcher trait, shaped like the one half_join had to write - #844

Merged
frankmcsherry merged 1 commit into
master-nextfrom
one-batcher-trait
Aug 26, 2026
Merged

One Batcher trait, shaped like the one half_join had to write#844
frankmcsherry merged 1 commit into
master-nextfrom
one-batcher-trait

Conversation

@frankmcsherry

Copy link
Copy Markdown
Member

dogsdogsdogs::half_join declared its own Batcher trait, because trace::Batcher did not fit a consumer that stages updates in a form of its own and releases them by a total order rather than an antichain. Two traits with the same name and the same job, one of them half the size. This adopts the smaller one and deletes the duplicate.

pub trait Batcher<T, C0, C1> {
    fn insert(&mut self, container: C0);
    fn extract<'a>(&'a mut self, upper: AntichainRef<'_, T>)
        -> (Option<C1>, AntichainRef<'a, T>);
}

Four consequences, in rough order of how much they matter.

Input and output types are now distinct. trace::Batcher pinned them to one associated type, which is why arrange_core carried a chunker whose container had to equal Ba::Output, and why arrange needed a doc caveat telling you to call arrange_core directly when they differ. A batcher may now release whatever its consumer means by a batch. half_join's is a sequence of chunks; arrange's is a chain it still hands to a Builder, which a following commit narrows to the batch itself.

Construction leaves the trait. This is the piece that looks like churn and isn't. A trait meant to cover both a merge batcher and — later — a builder-shaped one cannot mandate a new(logger, operator_id) constructor, because only the arrange path has a differential logger or a timely operator id to give. arrange_core takes the constructor as an argument, and MergeBatcher::new becomes inherent. Call sites move the batcher from turbofish to argument position:

// before
.arrange::<ValBatcher<_,_,_,_>, ValBuilder<_,_,_,_>, ValSpine<_,_,_,_>>()
// after
.arrange::<_, ValBuilder<_,_,_,_>, ValSpine<_,_,_,_>>(ValBatcher::new)

The Description leaves too. MergeBatcher::lower existed only to manufacture one, duplicating arrange_core's prev_frontier. The operator now builds the description from the frontiers it already tracks, and the field is gone.

seal and frontier collapse into one call, so the invariant between them — "the frontier of elements remaining after the most recent call to seal" — no longer needs stating. The retained frontier is reported by the extraction that determines it.

One behaviour change

The empty-batch branch used to call seal and discard the result, to advance the lower bound that no longer exists. Dropping that call also drops the full chain merge it performed on every empty frontier advance. Correctness is unaffected: no held capability precedes the input frontier in that branch, so no update does either, and nothing was extractable. It is the one thing here a benchmark could notice, in either direction.

+181 −194. Whole workspace builds warning-free and the test suite passes.

🤖 Generated with Claude Code

`dogsdogsdogs::half_join` declared its own `Batcher` trait, because
`trace::Batcher` did not fit a consumer that stages updates in a form of
its own and releases them by a total order rather than an antichain. Two
traits with the same name and the same job, one of them half the size.
This adopts the smaller one, and deletes the duplicate.

    pub trait Batcher<T, C0, C1> {
        fn insert(&mut self, container: &mut C0);
        fn extract<'a>(&'a mut self, upper: AntichainRef<'_, T>)
            -> (Option<C1>, AntichainRef<'a, T>);
    }

Four consequences, in rough order of how much they matter.

Input and output types are now distinct. `trace::Batcher` pinned them to
one associated type, which is why `arrange_core` carried a chunker whose
container had to equal `Ba::Output`. A batcher may now release whatever
its consumer means by a batch: half_join's is a sequence of chunks, and
arrange's is a chain it still hands to a `Builder`, which a following
commit narrows to the batch itself.

Construction leaves the trait. It has to: a trait meant to cover both a
merge batcher and, later, a builder-shaped one cannot mandate a
`new(logger, operator_id)` constructor, since only the arrange path has
a differential logger or an operator id to give. `arrange_core` takes the
constructor as an argument and `MergeBatcher::new` is inherent. Call
sites move the batcher from turbofish to argument position and are
otherwise unchanged.

The `Description` leaves too. `MergeBatcher::lower` existed only to
manufacture one, duplicating `arrange_core`'s `prev_frontier`; the
operator now builds it from the frontiers it already tracks.

`seal` and `frontier` collapse into one call, so the ordering invariant
between them ("the frontier after the most recent seal") is gone. The
retained frontier is reported by the extraction that determines it.

`insert` takes its container by reference, as `Builder::push` does, and
the implementor decides whether to claim the allocation or drain it. The
recycling decision belongs there rather than at the call site, which no
longer writes `std::mem::take`; `arrange_core` drops a `Default` bound as
a result. This also keeps the door open for a builder-shaped batcher:
`reduce` allocates one buffer outside its per-key loop and hands it over
by reference every key, which a by-value `insert` would cost it.

One behaviour change worth naming: the empty-batch branch used to call
`seal` and discard the result, to advance the lower bound that no longer
exists. Dropping that call also drops the full chain merge it performed
on every empty frontier advance. No held capability precedes the input
frontier there, so no update does either, and nothing was extractable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@frankmcsherry
frankmcsherry merged commit 0c4898f into master-next Aug 26, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant