Sub-task of #479. Read side, cgns backend — hardest read task. Covers part ④ of the read-side analysis (issuecomment-5166383239).
Goal
Allow cgns/reader.py::init_datasetdict_from_disk to read from a remote target.
Current state (verified on main)
init_datasetdict_from_disk builds CGNSDataset(path).
CGNSDataset.__getitem__ → Sample.load_from_dir → Sample.load → CGM.load(str(meshes_dir / "mesh_*.cgns")).
- pyCGNS (
CGNS.MAP.load) reads the CGNS/HDF5 file through a native C file handle — no fsspec object accepted.
- Split discovery also uses
local_path.iterdir() (same local-only issue as the zarr reader).
- No streaming reader exists for cgns today.
Options (design choice, not a config flag)
- (a) Materialize-to-temp: download remote sample dirs into a local temp dir, then reuse the existing local reader unchanged. Simpler, bounded temp usage (~1 sample at a time if done lazily in
__getitem__).
- (b) fsspec rewrite: rework
Sample.load around fsspec. More invasive, touches containers/sample.py.
Recommend evaluating (a) first (lazy per-sample temp in __getitem__) as it isolates the change to the cgns reader.
Depends on
Tests
memory:// or local-temp-backed source: verify a sample loads correctly.
Priority
★★★ (4/5) — most invasive read task; do after zarr read (#486) and constants (#487).
Sub-task of #479. Read side,
cgnsbackend — hardest read task. Covers part ④ of the read-side analysis (issuecomment-5166383239).Goal
Allow
cgns/reader.py::init_datasetdict_from_diskto read from a remote target.Current state (verified on
main)init_datasetdict_from_diskbuildsCGNSDataset(path).CGNSDataset.__getitem__→Sample.load_from_dir→Sample.load→CGM.load(str(meshes_dir / "mesh_*.cgns")).CGNS.MAP.load) reads the CGNS/HDF5 file through a native C file handle — no fsspec object accepted.local_path.iterdir()(same local-only issue as the zarr reader).Options (design choice, not a config flag)
__getitem__).Sample.loadaround fsspec. More invasive, touchescontainers/sample.py.Recommend evaluating (a) first (lazy per-sample temp in
__getitem__) as it isolates the change to the cgns reader.Depends on
fs.ls) shared with the zarr reader approach in [fsspec-aware #479] zarr: make init_from_disk read path fsspec-aware (read, priority 2) #486.Tests
memory://or local-temp-backed source: verify a sample loads correctly.Priority
★★★ (4/5) — most invasive read task; do after zarr read (#486) and constants (#487).