Skip to content

chore: improve reprovider with batch queue reads#1173

Open
gammazero wants to merge 3 commits into
mainfrom
provider-batch-read
Open

chore: improve reprovider with batch queue reads#1173
gammazero wants to merge 3 commits into
mainfrom
provider-batch-read

Conversation

@gammazero

Copy link
Copy Markdown
Contributor

Improve reprovider queue efficiency by attempting a batch read whenever more data is available.

Improve reprovider queue efficiency by attempting a batch read whenever more data is available.
@gammazero gammazero requested a review from a team as a code owner June 8, 2026 09:40
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 41.66667% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.56%. Comparing base (a88ea29) to head (4e79204).

Files with missing lines Patch % Lines
provider/reprovider.go 41.66% 5 Missing and 2 partials ⚠️

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1173      +/-   ##
==========================================
- Coverage   63.58%   63.56%   -0.02%     
==========================================
  Files         268      268              
  Lines       26964    26972       +8     
==========================================
+ Hits        17144    17145       +1     
- Misses       8102     8110       +8     
+ Partials     1718     1717       -1     
Files with missing lines Coverage Δ
provider/reprovider.go 57.74% <41.66%> (-0.59%) ⬇️

... and 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread provider/reprovider.go
provideOperation(s.ctx, c)
}

const batchReadSize = 2048

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intentional that this is the same number as dsqueue dedup cache size?

s.q = dsqueue.New(s.ds, "provide", dsqueue.WithDedupCacheSize(2048))

If yes, we might as well define a global const used at both places.

@lidel lidel mentioned this pull request Jun 12, 2026
23 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants