Sub-task of #479. Read side, transverse metadata layer — medium. Covers part ③ of the read-side analysis (issuecomment-5166383239).
Goal
Allow common/reader.py::load_constants_from_disk to read constants from a remote target.
Current state (verified on main)
- Backed by
np.memmap over data.mmap (load_constants_from_disk, l.172).
np.memmap requires a real seekable local file — passing an fsspec file handle fails with UnsupportedOperation: fileno (checked empirically).
_materialize_memmaps already exists to copy memmaps into plain in-memory arrays, but the source still has to be a local file.
Constraint
Remote constants cannot be read in place. The data.mmap bytes must land on local disk (temp) first, then np.memmap + _materialize_memmaps as today.
Scope
- When the source is remote (fsspec), stage
data.mmap (and companions) to a local temp file, memmap from there, materialize, then discard the temp.
- Keep the local fast path unchanged (no temp copy when already local).
Tests
memory:// source: verify constants load correctly via the staging path.
Priority
★★ (3/5) — transverse blocker shared by all backends on the read side; needed before cgns remote read is meaningful.
Sub-task of #479. Read side, transverse metadata layer — medium. Covers part ③ of the read-side analysis (issuecomment-5166383239).
Goal
Allow
common/reader.py::load_constants_from_diskto read constants from a remote target.Current state (verified on
main)np.memmapoverdata.mmap(load_constants_from_disk, l.172).np.memmaprequires a real seekable local file — passing an fsspec file handle fails withUnsupportedOperation: fileno(checked empirically)._materialize_memmapsalready exists to copy memmaps into plain in-memory arrays, but the source still has to be a local file.Constraint
Remote constants cannot be read in place. The
data.mmapbytes must land on local disk (temp) first, thennp.memmap+_materialize_memmapsas today.Scope
data.mmap(and companions) to a local temp file, memmap from there, materialize, then discard the temp.Tests
memory://source: verify constants load correctly via the staging path.Priority
★★ (3/5) — transverse blocker shared by all backends on the read side; needed before cgns remote read is meaningful.