Summary
ossfs's cat_file does not implement the range semantics defined by
fsspec.AbstractFileSystem.cat_file, where start/end behave "like usual
python slices", i.e. [start, end) with end exclusive and negative values
counting from EOF. Instead, both OSSFileSystem.cat_file and
AioOSSFileSystem._cat_file forward (start, end) unmodified to
oss2's get_object(byte_range=...), which serializes them into an HTTP
Range: bytes=start-end header — where end is inclusive (RFC 9110).
Actual vs expected behavior
For a file with contents data:
| call |
fsspec contract |
ossfs actual |
cat_file(p, 10, 20) |
data[10:20] (10 bytes) |
Range: bytes=10-20 → data[10:21] (11 bytes) |
cat_file(p, None, 500) |
data[:500] (first 500 bytes) |
Range: bytes=-500 → data[-500:] (last 500 bytes) |
cat_file(p, -100, None) |
data[-100:] |
correct by accident (HTTP suffix range) |
cat_file(p, 0, -5) |
data[:-5] |
Range: bytes=0--5 → invalid header |
The second row is the most damaging: silently wrong bytes, not just a wrong length.
This breaks consumers that stitch byte ranges together, e.g. kerchunk /
ReferenceFileSystem-style workloads, which rely on
cat_file(path, s, e) == data[s:e].
The same translation gap exists in OSSFileSystem.get_object() and
OSSFile._fetch_range (https://github.com/fsspec/ossfs/blob/main/src/ossfs/file.py),
which forward the exclusive end expected by AbstractBufferedFile's caches to the
inclusive OSS API (fetches one extra byte per block; mostly masked by the
readahead cache's final slice, but still incorrect).
Reproducer
import fsspec
content = bytes(range(256)) * 4 # 1024 bytes
fs = fsspec.filesystem("oss", ...) # any bucket
fs.pipe_file("my-bucket/range-test.bin", content)
for s, e in [(0, 100), (10, 20), (None, 500), (50, None), (-100, None), (0, -5), (7, 7)]:
got = fs.cat_file("my-bucket/range-test.bin", s, e)
expected = content[s:e]
assert got == expected, f"start={s}, end={e}: got {len(got)} bytes, expected {len(expected)}"
All positive-range cases return one extra byte; (None, 500) returns the last 500
bytes; (0, -5) produces an invalid header.
For comparison, s3fs, gcsfs, ocifs and fsspec's own http implementation all
translate to inclusive-end correctly (end - 1 after normalizing None/negatives).
Suggested fix
Normalize in cat_file / _cat_file before calling oss2, mirroring
_process_limits:
- resolve negative
start/end against the object size (self.info(path)["size"]),
- default
start to 0 when end is given,
- return
b"" when end <= start,
- call
get_object(byte_range=(start, end - 1)).
And in OSSFile._fetch_range, pass end - 1 as the inclusive end.
Happy to open a PR along these lines, including a parametrized conformance test
like the snippet above.
Environment
- ossfs: current main (core.py
cat_file as of bda100d)
- fsspec: 2026.x
Summary
ossfs'scat_filedoes not implement the range semantics defined byfsspec.AbstractFileSystem.cat_file, wherestart/endbehave "like usualpython slices", i.e.
[start, end)withendexclusive and negative valuescounting from EOF. Instead, both
OSSFileSystem.cat_fileandAioOSSFileSystem._cat_fileforward(start, end)unmodified tooss2'sget_object(byte_range=...), which serializes them into an HTTPRange: bytes=start-endheader — whereendis inclusive (RFC 9110).AbstractFileSystem.cat_file→f.read(end - f.tell()))AsyncFileSystem._process_limits, noteend -= 1 # bytes range is inclusive)OSSFileSystem.cat_file→byte_range=(start, end))AioOSSFileSystem._cat_file, same pattern)Actual vs expected behavior
For a file with contents
data:cat_file(p, 10, 20)data[10:20](10 bytes)Range: bytes=10-20→data[10:21](11 bytes)cat_file(p, None, 500)data[:500](first 500 bytes)Range: bytes=-500→data[-500:](last 500 bytes)cat_file(p, -100, None)data[-100:]cat_file(p, 0, -5)data[:-5]Range: bytes=0--5→ invalid headerThe second row is the most damaging: silently wrong bytes, not just a wrong length.
This breaks consumers that stitch byte ranges together, e.g.
kerchunk/ReferenceFileSystem-style workloads, which rely oncat_file(path, s, e) == data[s:e].The same translation gap exists in
OSSFileSystem.get_object()andOSSFile._fetch_range(https://github.com/fsspec/ossfs/blob/main/src/ossfs/file.py),which forward the exclusive
endexpected byAbstractBufferedFile's caches to theinclusive OSS API (fetches one extra byte per block; mostly masked by the
readahead cache's final slice, but still incorrect).
Reproducer
All positive-range cases return one extra byte;
(None, 500)returns the last 500bytes;
(0, -5)produces an invalid header.For comparison,
s3fs,gcsfs,ocifsand fsspec's ownhttpimplementation alltranslate to inclusive-end correctly (
end - 1after normalizingNone/negatives).Suggested fix
Normalize in
cat_file/_cat_filebefore calling oss2, mirroring_process_limits:start/endagainst the object size (self.info(path)["size"]),startto0whenendis given,b""whenend <= start,get_object(byte_range=(start, end - 1)).And in
OSSFile._fetch_range, passend - 1as the inclusive end.Happy to open a PR along these lines, including a parametrized conformance test
like the snippet above.
Environment
cat_fileas of bda100d)