Skip to content

Fix concurrent upload review issues#1554

Merged
TorstenDittmann merged 4 commits into
masterfrom
fix/concurrent-upload-review-feedback
May 21, 2026
Merged

Fix concurrent upload review issues#1554
TorstenDittmann merged 4 commits into
masterfrom
fix/concurrent-upload-review-feedback

Conversation

@TorstenDittmann
Copy link
Copy Markdown
Contributor

@TorstenDittmann TorstenDittmann commented May 21, 2026

What

  • Track the latest successful chunk response across SDK upload templates instead of relying on completion markers only
  • Fix PHP curl_multi handling and ensure cURL handles are cleaned up on exceptions
  • Remove unsafe concurrent uploadId/result writes in JVM/.NET paths and serialize progress callback invocation for JVM SDKs
  • Fix path file handle leaks in Python/.NET and reduce Dart/Flutter RandomAccessFile churn by opening one handle per worker
  • Stop Go/Ruby workers earlier on upload failures and clean up minor review issues
  • Adjust TS client ping return typing where non-JSON responses are wrapped\n\n

Verification

  • Regenerated affected SDKs with example.php
  • composer lint-twig
  • git diff --check

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 21, 2026

Greptile Summary

This PR fixes a suite of concurrency correctness issues across SDK upload templates — tracking the latest successful chunk response instead of relying solely on a completion marker, serializing progress callbacks in JVM SDKs, stopping workers early on error in Go/Ruby, and plugging file-handle/cURL-handle leaks in PHP, Python, and .NET.

  • Result tracking: All affected SDKs now maintain both a lastResponse/lastResult (updated on every chunk) and a completedResponse/completedResult (set only when the server confirms completion), returning the latter when available and falling back to the former — fixing the case where the completion-marked chunk is not the last one received by the consumer.
  • Resource cleanup: PHP cURL handles are now always removed and closed in a finally block; Python path-type files are opened with a context manager only when needed; .NET removes the leaked OpenRead stream on the path branch for single-chunk files; Dart/Flutter IO opens one RandomAccessFile per worker (closed in finally) instead of one per chunk.
  • Early exit on error: Go adds a done channel so the job-sender and idle workers stop immediately on the first chunk error; Ruby workers now break and store the first error instead of completing the queue, then join + re-raise after all threads finish.

Confidence Score: 5/5

All changes are targeted, well-scoped bug fixes with no new APIs beyond opt-in configuration properties; safe to merge.

Every changed file addresses a concrete defect (handle leak, unsafe concurrent write, missing early-exit, or wrong result selection) without introducing new behavioral contracts. The Go done channel and buffered results channel are correctly sized so in-flight workers can always drain without blocking after an early return. The PHP finally block unconditionally cleans up all cURL handles. JVM AtomicReference usage and synchronized(progressLock) are correct after awaitAll. .NET ReadAllBytes is only called for single-chunk uploads. No issues found that would cause incorrect behavior in production.

No files require special attention.

Important Files Changed

Filename Overview
mock-server/app/http.php Replaces non-atomic file_put_contents with fopen/flock/fwrite inside a try/finally that always closes the handle; adds CURLM_CALL_MULTI_PERFORM to the multi-exec loop condition
templates/go/client.go.twig Adds a done channel so workers stop accepting new jobs on the first chunk error; buffered results channel sized to remainingChunks prevents goroutine leaks on early return
templates/dotnet/Package/Client.cs.twig Removes the leaked OpenRead stream on the path branch (now uses ReadAllBytes only for sub-chunk-size files); multi-chunk path already uses ReadChunkAsync with using blocks; adds lastResult/completedResult tracking
templates/kotlin/src/main/kotlin/io/appwrite/Client.kt.twig Introduces AtomicReference for lastResultRef/completedResultRef and synchronizes onProgress callback; removes redundant uploadId == null guard; reads final result after awaitAll
templates/php/base/requests/file.twig Wraps curl_multi loop in try/finally ensuring all cURL handles are removed and closed on exception; adds CURLM_CALL_MULTI_PERFORM handling; separates lastResponse/completedResponse tracking
templates/python/package/client.py.twig Removes persistent file handle for path-type inputs (opens with context manager only when needed); adds final_result/last_result separation; uses result.get("$id") defensively
templates/ruby/lib/container/client.rb.twig Changes workers.each(&:value) to .join with explicit first_error re-raise; adds early break on error and last_result/completed_result tracking
templates/dart/lib/src/client_io.dart.twig Opens one RandomAccessFile per worker (closed in finally) instead of per-chunk; adds finalResponse/lastResponse separation
templates/rust/src/client.rs.twig Adds max_concurrency to UploadOptions; tracks completed_response separately from last_response; returns completed_response.or(last_response)
templates/node/src/client.ts.twig Removes unused index field from chunks array; adds finalResponse/lastResponse separation; changes ping() return type to Promise

Reviews (4): Last reviewed commit: "Close mock results handle on lock failur..." | Re-trigger Greptile

Comment thread templates/node/src/client.ts.twig
Comment thread templates/go/client.go.twig
Comment thread mock-server/app/http.php Outdated
@TorstenDittmann TorstenDittmann merged commit 9d85bde into master May 21, 2026
56 checks passed
@TorstenDittmann TorstenDittmann deleted the fix/concurrent-upload-review-feedback branch May 21, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants