[Feature]: Data Publishing & Published Datasets

[prbatero]

Problem Statement

Analysts and partners repeatedly ask "where is the final dataset for this event?" — today the answer is a set of ad-hoc publications.

HASTE lets you download a model's outputs (the damage GeoPackage, valid-area mask, building footprints, processed COGs, assessment report), but those downloads are ephemeral: they're SAS links tied to a single model that may be re-run or deleted, and there's no catalog of finished, curated outputs. There is no way to promote a result into a stable, named, described, discoverable dataset — nor to push HASTE outputs directly into the broader geospatial (STAC) ecosystem so partners can find and consume them with tools they already use.

Proposed Solution

Add a Published Datasets section (a sibling to the Model Catalog) plus a "Publish dataset" action on model results. Publishing routes through a provider abstraction so a dataset can be published to different targets:

• Local (HASTE storage): copies the selected artifacts into an immutable published/{datasetId}/ prefix — independent of the source model's lifecycle — and exposes stable retrieval links.
• Planetary Computer (MPC Pro GeoCatalog): builds STAC (one collection per project/event; item geometry from the valid-area mask) and ingests it, storing the collection id + explorer/collection links on the record.

Highlights:

• Per-asset selection in the publish dialog (damage GeoPackage, valid mask, building footprints, image COG) — publish exactly what you intend to share.
• Async publish job with visible status (PENDING → IN_PROGRESS → PUBLISHED / FAILED), matching the existing training/inference/zip pattern.
• Extensible by design: a new target is a new PublishingProvider subclass + a registry entry — no UI/API/queue changes.
• Operator-owned config: provider settings are Azure App Settings + managed identity (set at deploy); no in-app admin UI in v1. The dialog only reflects isConfigured.
• PC v1 is download-only: in the Explorer a published dataset shows as item footprints + metadata, and the GeoPackage is downloaded (PC Pro tiles raster only, not vector). Rasterized-COG rendering is a future enhancement.

Alternatives Considered

• Keep relying on per-model download links — rejected: ephemeral, not discoverable, no provenance or stable identity.
• Local-only publishing without a provider abstraction — rejected: extensibility to Planetary Computer (and future STAC/portal targets) is a core requirement, so the seam must exist from day one.
• Use the azure-planetarycomputer SDK for the PC target — instead the design uses a small vendored REST GeoCatalog client, which is dependency-light and fully unit-testable against HTTP fixtures.

Area

Core library (hastelib)

Additional Context

Full feature specification lives in spec/features/data-publishing/ (merged via #112) — includes user stories, UX spec, technical design, data model, plan, impact analysis, test plan, and rollout.

Modeled on the existing Model Catalog pattern (single index metadata doc, Get/Put/Delete routes, catalog-style React page).

Components affected:

• hastelib: new core/publishing/ package (provider ABC, registry, Local + Planetary Computer providers, STAC builder), models/publishing.py, processors/publishing.py, publishing config keys.
• API: GetPublishingProviders, GetPublishedDatasets, GetPublishedDataset, PutPublishDatasetQueueMessage, DeletePublishedDataset.
• Queues: GetPublishDatasetQueueMessage trigger on a new publish-queue.
• UI: PublishedDatasets section, PublishDatasetModal, and a "Publish dataset…" entry on model results.
• Infra: publishing App Settings + feature flags (PUBLISHING_ENABLED, PC_PROVIDER_ENABLED, PC_GEOCATALOG_URL, …).

Implementation is in progress on prbatero/feat/data-publishing-local (Local target) with a follow-up branch for the Planetary Computer provider.

Checklist

• I have searched existing issues and this has not already been requested.