Skip to content

Commit pre-built adjacency zarrs to repo for out-of-the-box routing #176

Description

@taddyb

Context

After #172, the adjacency zarrs contain everything needed for routing (topology + flowpath attributes). These files are small enough (<50 MB each) to live in the repository, eliminating the need for any manual data preparation before running DDR.

Current sizes (topology only):

  • hydrofabric_v2.2_conus_adjacency.zarr — 5.8 MB
  • merit_conus_adjacency.zarr — 2.3 MB

With baked-in flowpath attributes, these will grow modestly (5 float arrays × ~300-800k segments × 4 bytes ≈ 6-16 MB additional).

Proposal

  1. Rebuild both adjacency zarrs using the updated engine scripts from Bake flowpath attributes into adjacency zarr at engine build time #172
  2. Commit them to data/ (or examples/data/) in the repository
  3. Update example configs to point to the in-repo zarrs with relative paths
  4. Add to .gitattributes if needed for LFS (likely unnecessary at <50 MB)

Per-gage adjacency zarrs

The *_gages_conus_adjacency.zarr files are ~300 MB each — too large for the repo. These are only needed for gage-based training/testing, not for routing. They should remain a separate download (future ddr fetch or manual).

Files to modify

  • data/ or examples/data/ — add zarr stores
  • examples/merit/example_config.yaml — update paths
  • examples/lynker_hydrofabric/example_config.yaml — update paths
  • config/templates/*.yaml — update default paths

Acceptance criteria

  • Both adjacency zarrs committed and under 50 MB each
  • ddr route --config-name example resolves the adjacency zarr without any downloads
  • .gitignore updated if needed to not exclude these specific zarrs
  • Repo clone size remains reasonable

Depends on

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions