Skip to content

Keep requests session open in S3Streamer - #41

Open
rapto wants to merge 1 commit into
ecmwf:developfrom
meteogrid:keep-session-in-s3
Open

Keep requests session open in S3Streamer#41
rapto wants to merge 1 commit into
ecmwf:developfrom
meteogrid:keep-session-in-s3

Conversation

@rapto

@rapto rapto commented Feb 25, 2026

Copy link
Copy Markdown

Description

When downloading many chunks of files, a supervisor thread is run for each of them, causing 'too many open files' error. This patch uses only one session per S3 instance.

Contributor Declaration

By opening this pull request, I affirm the following:

  • All authors agree to the Contributor License Agreement.
  • The code follows the project's coding standards.
  • I have performed self-review and added comments where needed.
  • I have added or updated tests to verify that my changes are effective and functional.
  • I have run all existing tests and confirmed they pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant