An easy way to demonstrate the issue is to start with examples/http_get2.zig with a website which gives chunked responses (Transfer-Encoding: chunked). I'm using https://datatracker.ietf.org/doc/html/rfc9112 as an example.
Running latest code (0.15.2 branch) seems to be working:
Header length: 1104, content length: 0, chunked: true
chunk 24570 bytes ... m-check
chunk 24927 bytes ... /html>
chunked in 2 chunks 49497 bytes
Now, suppose we want to collect the response body in an allocating writer. Let's create it before the loop:
var body = Writer.Allocating.init(gpa);
defer body.deinit();
and replace existing code
|
const chunk_body = try rdr.readAlloc(gpa, cp.chunk_len); |
|
defer gpa.free(chunk_body); |
with
try rdr.streamExact(&body.writer, cp.chunk_len);
It fails with the following:
header length: 1114, content length: 0, chunked: true
error: EndOfStream
/home/oleg/.cache/zig/p/tls-0.1.0-ER2e0iRoBQAUWlLfs_azUOsDWlbvYnBr_tS-rkR5Chw0/src/connection.zig:261:25: 0x14a2817 in stream (root.zig)
if (n == 0) return error.EndOfStream;
^
/home/oleg/.local/zig-0.15.2/lib/std/Io/Reader.zig:178:15: 0x109bfc2 in stream (std.zig)
const n = try r.vtable.stream(r, w, limit);
^
/home/oleg/.local/zig-0.15.2/lib/std/Io/Reader.zig:215:41: 0x1272c92 in streamExact (std.zig)
while (remaining != 0) remaining -= try r.stream(w, .limited(remaining));
^
/home/oleg/Projects/tls-test/src/main.zig:118:17: 0x11e5baf in readHttpResponse (main.zig)
try rdr.streamExact(&body.writer, cp.chunk_len);
^
/home/oleg/Projects/tls-test/src/main.zig:52:5: 0x11e739d in main (main.zig)
try readHttpResponse(gpa, &conn);
^
I believe the problem is related to tls.Connection.Reader.stream function:
|
fn stream(r: *Io.Reader, w: *Io.Writer, limit: Io.Limit) Io.Reader.StreamError!usize { |
|
const self: *Reader = @fieldParentPtr("interface", r); |
|
const n = self.conn.read(limit.slice(w.unusedCapacitySlice())) catch |err| { |
|
self.err = err; |
|
if (err == error.EndOfStream) return error.EndOfStream; |
|
return error.ReadFailed; |
|
}; |
|
if (n == 0) return error.EndOfStream; |
|
w.advance(n); |
|
return n; |
|
} |
Current implementation calls
w.unusedCapacitySlice() and then
w.advance(n) so it just fills data buffer but does not call actual Writer's logic producing EndOfStream error once the buffer is full.
Using writableSliceGreedy() solves the problem (that's what Zig standard library uses in std.fs.File.Reader for instance).
An easy way to demonstrate the issue is to start with
examples/http_get2.zigwith a website which gives chunked responses (Transfer-Encoding: chunked). I'm using https://datatracker.ietf.org/doc/html/rfc9112 as an example.Running latest code (0.15.2 branch) seems to be working:
Now, suppose we want to collect the response body in an allocating writer. Let's create it before the loop:
and replace existing code
tls.zig/example/http_get2.zig
Lines 106 to 107 in 4770c40
with
It fails with the following:
I believe the problem is related to
tls.Connection.Reader.streamfunction:tls.zig/src/connection.zig
Lines 254 to 264 in 4770c40
Current implementation calls
w.unusedCapacitySlice()and thenw.advance(n)so it just fills data buffer but does not call actual Writer's logic producing EndOfStream error once the buffer is full.Using
writableSliceGreedy()solves the problem (that's what Zig standard library uses instd.fs.File.Readerfor instance).