Summary
When doProcessors reports errors on the error channel, it loses two distinct pieces of information:
- Per-item errors and processor-wide errors are indistinguishable. Both are wrapped as
ProcessorError, so a caller using errors.As(err, &pe) cannot tell whether a whole processing stage failed or whether a single item failed within an otherwise-healthy batch.
- The failing item's
ID is thrown away. A per-item error is wrapped as &ProcessorError{Err: item.Error} with no reference to the Item it came from, so callers can't correlate the error back to the input item.
This contradicts the documented contract in CLAUDE.md ("Items with errors are tracked individually through the Error field") — the per-item tracking exists right up until the error hits the channel, where the ID is dropped and the error is relabeled as a generic stage error.
Where
batch/batch.go, in doProcessors:
var err error
items, err = proc.Process(ctx, items)
if err != nil {
b.errs <- &ProcessorError{Err: err} // batch.go:424 — processor-WIDE error
}
// ...
for _, item := range items {
if item.Error != nil {
b.errs <- &ProcessorError{Err: item.Error} // batch.go:430 — PER-ITEM error, ID lost
}
}Both sends produce the same ProcessorError type, and the second discards item.ID.
Impact
- A caller cannot route/retry/dead-letter a specific failed item, because it has no idea which item (by
ID) failed.
- A caller cannot distinguish "stage N blew up" from "item 4,217 had a validation error" — both surface identically as
processor error: ....
- Defeats the purpose of
Item carrying both a unique ID and a per-item Error field.
Possible directions
(Design decision — listing options, not prescribing one.)
- Introduce a distinct
ItemError type that carries the ID (and optionally the offending Item/Data), e.g.:
type ItemError struct {
ItemID uint64
Err error
}
func (e ItemError) Error() string { return fmt.Sprintf("item %d error: %v", e.ItemID, e.Err) }
func (e ItemError) Unwrap() error { return e.Err }&ItemError{ItemID: item.ID, Err: item.Error} from the per-item path while keeping ProcessorError for stage-wide failures.
- Alternatively, attach the
ID to ProcessorError (optional field, zero when stage-wide) so existing errors.As(err, &ProcessorError{}) callers keep working.
Either way: stage-wide vs per-item should be distinguishable, and the per-item path should preserve Item.ID.
Notes
References
Summary
When
doProcessorsreports errors on the error channel, it loses two distinct pieces of information:ProcessorError, so a caller usingerrors.As(err, &pe)cannot tell whether a whole processing stage failed or whether a single item failed within an otherwise-healthy batch.IDis thrown away. A per-item error is wrapped as&ProcessorError{Err: item.Error}with no reference to theItemit came from, so callers can't correlate the error back to the input item.This contradicts the documented contract in
CLAUDE.md("Items with errors are tracked individually through theErrorfield") — the per-item tracking exists right up until the error hits the channel, where theIDis dropped and the error is relabeled as a generic stage error.Where
batch/batch.go, indoProcessors:Both sends produce the same
ProcessorErrortype, and the second discardsitem.ID.Impact
ID) failed.processor error: ....Itemcarrying both a uniqueIDand a per-itemErrorfield.Possible directions
(Design decision — listing options, not prescribing one.)
ItemErrortype that carries theID(and optionally the offendingItem/Data), e.g.:&ItemError{ItemID: item.ID, Err: item.Error}from the per-item path while keepingProcessorErrorfor stage-wide failures.IDtoProcessorError(optional field, zero when stage-wide) so existingerrors.As(err, &ProcessorError{})callers keep working.Either way: stage-wide vs per-item should be distinguishable, and the per-item path should preserve
Item.ID.Notes
Filtersilently dropping errored items, and fix(batch): panic recovery, deadlock-safe error sends, bounded prealloc, Done() race #65 documents theerrors.Aspointer-target form — but neither addresses the conflation or the lostIDdescribed here.ItemErroris additive (existingProcessorErrormatches break for per-item errors only); adding a field toProcessorErroris fully backwards-compatible. v0, so either is acceptable with a CHANGELOG note.References
batch/batch.go—doProcessorsper-item vs processor-wide error sends.batch/errors.go— currentProcessorError/SourceErrordefinitions.errors.Asguidance), API hardening & cleanup (ROAST.md architectural review) #75 (API hardening tracker).