ACCESS CMIP7 Submission Tracker

Entry dashboard for the ACCESS CMIP7 submission workflow. Tracks progress through CMORisation, QC, and ESGF publication for ACCESS model submissions, with access-moppy-qc-registry serving as the companion QC registry.

Dashboard

→ Open dashboard

View	Description
Overview	Per-experiment cards with progress bars per ensemble member
Experiment Detail	Variable × member matrix for one experiment
Member Timeline	All variables for one (model, experiment, member) sorted by stage
Variable Pipeline	One variable across all (experiment, member) combinations, with links to inspect or suggest QC checks
CMOR Requests	GitHub issue-backed CMORisation work requests, plus planned combinations still missing retrospective request metadata

Pipeline stages

not_started → planned → cmorised ─┬→ qc_pending → qc_pass ──┬→ published
                                   └→ qc_fail                └→ qc_warn ─┘
                    failed (batch error)

Repository structure

plans/                          # Submission intent — one YAML per model
  ACCESS-ESM1.6.yaml
requests/                       # CMORisation work requests accepted from GitHub issues
  <model>_<experiment>_<member>.yaml
progress/                       # Ingested runtime reports
  <model>/
    <experiment>/
      <member>/
        cmorisation.json        ← from moppy_batch_report.json (via ingest_report.py)
        qc.json                 ← from MOPPy QC runner (future)
        publication.json        ← manually updated or ESGF API script
schemas/                        # JSON Schemas for validation
scripts/
  ingest_report.py              # Place a batch report into the hierarchy
  compile_progress.py           # Build dashboard/progress.json
  validate_plans.py             # Validate plans/*.yaml
dashboard/                      # Static GitHub Pages site
  index.html / style.css / app.js
  progress.json                 # Generated by CI — do not edit manually
.github/
  workflows/
    validate_plans.yml          # Run on every PR touching plans/ or progress/
    build_dashboard.yml         # Rebuild + deploy on merge to main

Adding a new model

1. Add plans/<Model-ID>.yaml following the schema in schemas/plan.schema.json.
2. Open a PR — CI will validate the file automatically.

Requesting CMORisation work

Open the GitHub issue form:

https://github.com/access-nri/access-moppy-progress/issues/new?template=propose_submission.yml

The form captures the experiment/member to CMORise, the Gadi path to the raw output, the parent experiment metadata needed by ACCESS-MOPPy, extra notes, and the best contact for follow-up questions.

When a maintainer applies the status/accepted label to the issue, CI will:

1. Convert the issue into requests/<model>_<experiment>_<member>.yaml.
2. Open a PR with the generated request file for review.
3. Surface the request in the dashboard CMOR Requests view.

That request view also highlights planned experiment/member combinations that still have no request record, so existing work can be backfilled retrospectively through the same issue workflow.

Ingesting a batch report

After a MOPPy run completes on the HPC:

# On NCI or locally, after copying the report file:
python scripts/ingest_report.py \
    --report /path/to/moppy_batch_report.json \
    --model  ACCESS-ESM1.6 \
    --member r1i1p1f1

# Then commit and push:
git add progress/
git commit -m "chore: ingest ACCESS-ESM1.6 historical/r1i1p1f1"
git push

CI will recompile progress.json and redeploy the dashboard automatically.

Updating publication status

Edit progress/<model>/<experiment>/<member>/publication.json directly. The file follows schemas/publication.schema.json. Commit and push — the dashboard updates on the next CI run.

Local development

python scripts/validate_plans.py
python scripts/compile_progress.py
cd dashboard && python -m http.server 8080

Dependencies: pyyaml, jsonschema (Python 3.10+).

With Pixi

This repo now includes a pixi.toml for a managed dev environment.

pixi run dev

That will:

1. validate plans/*.yaml
2. rebuild dashboard/progress.json
3. serve the dashboard at http://localhost:8080

Other handy commands:

pixi run validate-plans
pixi run validate-requests
pixi run compile-progress
pixi run serve-dashboard

Licence

MIT