From 05d73f67c168d1059dfd8108da97a1e19cc13af9 Mon Sep 17 00:00:00 2001 From: Vaughn Friesen Date: Fri, 29 May 2026 19:55:04 +0800 Subject: [PATCH 1/3] docs: correct Nil processor description and errors.As guidance Fix #8: the Nil processor was described as "passes items through unchanged (for benchmarking)", which is wrong. Per processor/nil.go it sleeps for a configurable Duration (and can mark items cancelled via MarkCancelled). Corrected in CLAUDE.md, AGENTS.md, and README.md to match processor/doc.go. Fix #9: the engine emits wrapped errors as pointers (&SourceError{}, &ProcessorError{} in batch/batch.go), so errors.As must target the pointer type. Made the CLAUDE.md/AGENTS.md guidance explicit with a concrete example; README already used the pointer form. Co-Authored-By: Claude Opus 4.8 (1M context) --- AGENTS.md | 14 +++++++------- CLAUDE.md | 14 +++++++------- README.md | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3c39118..7ae53c8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,11 +34,11 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo - Include practical examples in documentation ## File Structure -- `/batch`: Core batch processing functionality, includes the main Batch type, configuration, errors, and helper functions -- `/processor`: Different processors for data manipulation (Transform, Filter, Channel, Error, Nil) -- `/source`: Data sources for batch processing (Channel, Error, Nil) -- `/doc.go`: Package-level documentation -- `/example_test.go`: Top-level usage examples +- `/batch`: Core batch processing package. Includes `batch.go` (the main `Batch` type and pipeline), `config.go` (Config interface, ConstantConfig, DynamicConfig, BufferConfig), `errors.go` (SourceError, ProcessorError), `constants.go`, `helpers.go` (IgnoreErrors, CollectErrors, RunBatchAndWait, ExecuteBatches), and `doc.go`. Accompanied by unit tests (`*_test.go`) and several runnable `example_*_test.go` files. +- `/processor`: Processor implementations — `transform.go`, `filter.go`, `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests. +- `/source`: Source implementations — `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests. +- `/doc.go`: Root package-level documentation. +- `/example_test.go`: Top-level usage examples. ## Key Concepts - **Batch**: Main type that orchestrates the batch processing pipeline @@ -57,7 +57,7 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo - **Filter**: Filters items based on a predicate function - **Channel**: Writes item data to an output channel - **Error**: Simulates processor errors (for testing) -- **Nil**: Passes items through unchanged (for benchmarking) +- **Nil**: Sleeps for a configurable `Duration` without modifying item data (can mark items cancelled via `MarkCancelled`); useful for simulating slow processing and testing timing behavior ### Sources - **Channel**: Uses Go channels as data sources @@ -75,4 +75,4 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo - Processors should respect context cancellation - Items with errors are tracked individually through the Error field - Batch processing continues despite individual item errors -- Use `errors.As` to check error types (SourceError, ProcessorError) +- Use `errors.As` to check error types (SourceError, ProcessorError). The engine wraps errors as pointers (`&SourceError{...}`, `&ProcessorError{...}`), so `errors.As` must target the pointer type — e.g. `var se *batch.SourceError; errors.As(err, &se)` (equivalently `errors.As(err, new(*batch.SourceError))`). A value-target such as `var se batch.SourceError; errors.As(err, &se)` will not match. diff --git a/CLAUDE.md b/CLAUDE.md index ddc25b4..b8fc557 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -34,11 +34,11 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo - Include practical examples in documentation ## File Structure -- `/batch`: Core batch processing functionality, includes the main Batch type, configuration, errors, and helper functions -- `/processor`: Different processors for data manipulation (Transform, Filter, Channel, Error, Nil) -- `/source`: Data sources for batch processing (Channel, Error, Nil) -- `/doc.go`: Package-level documentation -- `/example_test.go`: Top-level usage examples +- `/batch`: Core batch processing package. Includes `batch.go` (the main `Batch` type and pipeline), `config.go` (Config interface, ConstantConfig, DynamicConfig, BufferConfig), `errors.go` (SourceError, ProcessorError), `constants.go`, `helpers.go` (IgnoreErrors, CollectErrors, RunBatchAndWait, ExecuteBatches), and `doc.go`. Accompanied by unit tests (`*_test.go`) and several runnable `example_*_test.go` files. +- `/processor`: Processor implementations — `transform.go`, `filter.go`, `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests. +- `/source`: Source implementations — `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests. +- `/doc.go`: Root package-level documentation. +- `/example_test.go`: Top-level usage examples. ## Key Concepts - **Batch**: Main type that orchestrates the batch processing pipeline @@ -57,7 +57,7 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo - **Filter**: Filters items based on a predicate function - **Channel**: Writes item data to an output channel - **Error**: Simulates processor errors (for testing) -- **Nil**: Passes items through unchanged (for benchmarking) +- **Nil**: Sleeps for a configurable `Duration` without modifying item data (can mark items cancelled via `MarkCancelled`); useful for simulating slow processing and testing timing behavior ### Sources - **Channel**: Uses Go channels as data sources @@ -75,4 +75,4 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo - Processors should respect context cancellation - Items with errors are tracked individually through the Error field - Batch processing continues despite individual item errors -- Use `errors.As` to check error types (SourceError, ProcessorError) \ No newline at end of file +- Use `errors.As` to check error types (SourceError, ProcessorError). The engine wraps errors as pointers (`&SourceError{...}`, `&ProcessorError{...}`), so `errors.As` must target the pointer type — e.g. `var se *batch.SourceError; errors.As(err, &se)` (equivalently `errors.As(err, new(*batch.SourceError))`). A value-target such as `var se batch.SourceError; errors.As(err, &se)` will not match. \ No newline at end of file diff --git a/README.md b/README.md index 56afdbc..34b0fed 100644 --- a/README.md +++ b/README.md @@ -100,7 +100,7 @@ go get github.com/MasterOfBinary/gobatch - **Filter**: Filters items based on a predicate function. - **Transform**: Transforms item data with a custom function. - **Error**: Simulates processor errors for testing. -- **Nil**: Passes items through unchanged for benchmarking. +- **Nil**: Sleeps for a configurable `Duration` without modifying item data; useful for simulating slow processing and testing timing behavior. - **Channel**: Writes item data to an output channel. ### Built-in Sources From a13e76b3d6661a952d123d4a80125c91a501294e Mon Sep 17 00:00:00 2001 From: Vaughn Friesen Date: Fri, 29 May 2026 19:55:15 +0800 Subject: [PATCH 2/3] docs: clean up doc.go example comments Root doc.go: removed the "// Output:" marker from the package doc comment. It was not a real runnable Example and could mislead readers into thinking the snippet's output was verified; the illustrative code is kept with a plain comment. source/doc.go: the example drained out fully and then errs sequentially. That only works because Channel emits no errors; added a caution that data and errors should generally be consumed concurrently, since the Batch engine reads both channels at the same time. Co-Authored-By: Claude Opus 4.8 (1M context) --- doc.go | 4 +--- source/doc.go | 12 +++++++----- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/doc.go b/doc.go index 74d1ff1..e96fb2c 100644 --- a/doc.go +++ b/doc.go @@ -26,9 +26,7 @@ // batch.IgnoreErrors(errs) // <-b.Done() // -// Output: -// -// hello +// The Transform func above prints each item, so this pipeline writes "hello". // // See the README.md for an overview of how these pieces fit together. package gobatch diff --git a/source/doc.go b/source/doc.go index 66bf44a..8dfc19c 100644 --- a/source/doc.go +++ b/source/doc.go @@ -18,13 +18,15 @@ // src := &Channel[int]{Input: input} // out, errs := src.Read(context.Background()) // for item := range out { -// fmt.Println(item) +// fmt.Println(item) // prints 1 then 2 // } // for range errs { // } // -// Output: -// -// 1 -// 2 +// Caution: draining out fully and then errs sequentially, as shown above, is +// only safe because Channel emits no errors. In general, data and errors should +// be consumed concurrently — the Batch engine reads both channels at the same +// time — otherwise a source that emits an error while data is still pending can +// block. Prefer separate goroutines (or a select) when a source may report +// errors. package source From 849789b1bc62d2a848f7640debf66be609f63af7 Mon Sep 17 00:00:00 2001 From: Vaughn Friesen Date: Sat, 30 May 2026 00:33:37 +0800 Subject: [PATCH 3/3] docs: reconcile doc corrections with single-use API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebased fix/doc-corrections onto master (ba2a756), which merged the single-use Batch API (#64) and the golangci-lint v2.12.2 install line (#78) into the same doc files this PR edits. The rebase auto-merged without textual conflicts, but doc.go required a genuine three-way reconciliation: master rewrote the package-doc snippet to the new API (errs, err := b.Go(...); if err != nil { log.Fatal(err) }; batch.IgnoreErrors(errs)) while this PR independently removed the misleading "// Output:" marker from that same block. Both edits were combined off the common ancestor, so the result keeps the new API and drops the marker — no manual conflict markers were ever produced. Reconciliation outcome: - Kept master's content: single-use semantics, Go's (<-chan error, error) signature, ErrBatchUsed/ErrNilSource, nil-Config default note, the removal of IDBufferSize, and the golangci-lint install command. - Re-applied this PR's still-valid corrections on top: Nil processor now documented as sleeping for a Duration (CLAUDE.md/AGENTS.md/README.md), errors.As pointer-target guidance (CLAUDE.md/AGENTS.md, verified against the &SourceError{}/&ProcessorError{} sends in batch/batch.go), File Structure refresh, and the doc.go / source/doc.go "// Output:" cleanups. - Dropped nothing as redundant: master and this PR touched disjoint concerns, so every correction this PR intended still applies. Verified post-rebase: go build ./..., go vet ./..., golangci-lint v2.12.2 run --timeout=3m (0 issues), go test ./... (all packages pass, including the runnable root Example), gofmt -l . empty. Co-Authored-By: Claude Opus 4.8 (1M context)