-
Notifications
You must be signed in to change notification settings - Fork 6
Data node API
Radu Marias edited this page Sep 1, 2024
·
4 revisions
The clients will communicate with data nodes with HTTP to upload/download files using Content-Range header. Alternatively we could offer accessing files with a BitTorrent client, Apache Arrow Flight with direct I/O, zero-copy using sendfile() to eliminate using OS's buffer and minimizing CPU usage.
We could also offer SFTP and FTPS as those are also accessing the file in parallel with resumable transfer.