Skip to content

Validate the EgoSuite label-report envelope before reading frames #307

Description

@kstonekuan

Current behavior

write_label_report() writes schema_version and label_type alongside the frame records.

load_projected_hand_label_report() checks that the payload is an object with a frames array, then starts parsing frame records. It never reads schema_version or label_type. The current happy-path test builds a report containing only frames at examples/egosuite_evaluation/tests/test_evaluation.py:148-179.

A report from another producer, an unrelated report type, or a future incompatible schema can therefore enter the current frame parser when its records happen to have the expected field names.

Verified against main at fbfd553.

Pattern to copy

The dataset snapshot marker parser validates both the format name and version before consuming the directory at src/hflow/snapshot.py:502-530. Its errors name the marker, expected format, and supported version.

What to build

Validate the saved label report's envelope before reading any frame record. The accepted envelope is the current SCHEMA_VERSION plus label_type projected-hand-joints.

Definition of done

  1. A report written by write_label_report() still loads unchanged.
  2. Missing, non-integer, older, and future schema_version values are refused before frame parsing.
  3. Missing, non-string, and unsupported label_type values are refused before frame parsing.
  4. Each error names the report path, the invalid field, the value found, and the supported value.
  5. The existing per-frame validation and ordering behavior stays unchanged.
  6. The happy-path fixture includes the real envelope.
  7. Add boundary cases for every rejected envelope shape without testing json library behavior.
  8. No new dependency is added.

Non-goals

  • Migrating unknown report versions
  • Changing projected-hand geometry or label semantics
  • Changing how a canonical episode is matched to its labels

Validation

uv run --locked --project examples/egosuite_evaluation \
  ruff check --fix examples/egosuite_evaluation
uv run --locked --project examples/egosuite_evaluation \
  ruff format examples/egosuite_evaluation
uv run --locked --project examples/egosuite_evaluation \
  ty check --project examples/egosuite_evaluation --extra-search-path . \
  examples/egosuite_evaluation
uv run --locked --project examples/egosuite_evaluation \
  pytest -q examples/egosuite_evaluation/tests
uv run pytest -q

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomershelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions