Current behavior
The LeRobot importer has typed top-level episode and video-window dictionaries, but _SourceArchive still contains untyped nested values at src/hflow/importers/lerobot.py:50-89:
info: dict
numeric_features: dict[str, dict]
_fetch_info_json() returns a raw dictionary at src/hflow/importers/lerobot.py:260-276. _ensure_source_archive() then parses fps, path templates, features, and feature kinds at several later points in src/hflow/importers/lerobot.py:300-469. Conversion continues to index the archive and inspect optional dictionaries at src/hflow/importers/lerobot.py:639-685.
Recent metadata bugs such as #299 and the boolean shape-dimension fix in fbfd553 both came from values remaining loosely typed after the external JSON boundary.
Verified against main at fbfd553.
Pattern to copy
parse_project_config() accepts an open external document, validates its supported fields once, and returns a ProjectConfig dataclass. DatasetSource in the importer already provides the same internal shape for repository identity.
What to build
Parse supported meta/info.json fields into immutable internal values before metadata discovery or conversion continues. Give the source archive, episode rows, video windows, and supported numeric feature specifications attribute-based domain types.
Keep the downloaded info.json cache as the original JSON-compatible document. Unknown upstream fields can remain ignored. Unsupported selected feature shapes should still fail before any episode or manifest is published.
This shares _ensure_source_archive() with #293 and #299. Preserve those fixes and rebase if either lands first.
Definition of done
- A dedicated parser turns raw
meta/info.json into a typed dataset-information value.
- FPS, data and video path templates, robot type, video feature names, and supported numeric feature shapes are validated at that boundary.
_ensure_source_archive() returns an immutable archive type, with no TypedDict containing nested bare dictionaries.
- Episode conversion consumes typed archive, episode, video-window, and numeric-feature values.
- Code after the parser does not call
.get() on raw LeRobot feature specifications.
- Existing valid synthetic v3 metadata produces the same selected episodes, paths, schemas, provenance, and manifest.
- Existing malformed FPS, dtype, shape, and required-feature refusals keep failing before dataset output.
- Unknown metadata fields remain compatible.
Non-goals
- Modeling every Hugging Face API or DuckDB row
- Supporting new LeRobot dtypes or feature ranks
- Changing the prepared manifest or canonical MCAP format
- Adding LeRobot, Pydantic, or another dependency
Validation
uv run ruff check --fix
uv run ruff format
uv run ty check
uv run pytest -q tests/test_lerobot_converter.py
uv run pytest -q
Current behavior
The LeRobot importer has typed top-level episode and video-window dictionaries, but
_SourceArchivestill contains untyped nested values atsrc/hflow/importers/lerobot.py:50-89:_fetch_info_json()returns a raw dictionary atsrc/hflow/importers/lerobot.py:260-276._ensure_source_archive()then parsesfps, path templates, features, and feature kinds at several later points insrc/hflow/importers/lerobot.py:300-469. Conversion continues to index the archive and inspect optional dictionaries atsrc/hflow/importers/lerobot.py:639-685.Recent metadata bugs such as #299 and the boolean shape-dimension fix in
fbfd553both came from values remaining loosely typed after the external JSON boundary.Verified against
mainatfbfd553.Pattern to copy
parse_project_config()accepts an open external document, validates its supported fields once, and returns aProjectConfigdataclass.DatasetSourcein the importer already provides the same internal shape for repository identity.What to build
Parse supported
meta/info.jsonfields into immutable internal values before metadata discovery or conversion continues. Give the source archive, episode rows, video windows, and supported numeric feature specifications attribute-based domain types.Keep the downloaded
info.jsoncache as the original JSON-compatible document. Unknown upstream fields can remain ignored. Unsupported selected feature shapes should still fail before any episode or manifest is published.This shares
_ensure_source_archive()with #293 and #299. Preserve those fixes and rebase if either lands first.Definition of done
meta/info.jsoninto a typed dataset-information value._ensure_source_archive()returns an immutable archive type, with noTypedDictcontaining nested bare dictionaries..get()on raw LeRobot feature specifications.Non-goals
Validation