ECCO-DarwinDiff

A differentiable PyTorch reimplementation of ECCO-Darwin ocean biogeochemistry — gradients flow through every step of the simulation, so one loss surface recovers the identifiable subset of the parameters that Green's-functions calibration tunes one at a time, predicted per grid cell from the local environment — a surrogate-to-model identifiability study.

Documentation · Project status · Results matrix · Quick start · Citation

ECCO-Darwin (Carroll et al. 2020, JAMES; 2022, GBC) calibrates its ocean-biogeochemistry parameters with Green's functions — a method that needs a fresh full forward run per parameter, so the published calibration tunes only six. ECCO-DarwinDiff replaces the biogeochemistry side with PyTorch autograd: gradients for every parameter in a single backward pass, with the values varying across space, predicted by a small per-cell network. The work is framed as a surrogate-to-model identifiability study — which of the six Carroll-6 parameters are identifiable from real ocean observations, which are not, and why. The study is complete; paper #1 is in preparation.

Two tracks, both now at a collaboration gate (awaiting a domain-expert read, not more compute): (1) Parameter learner (paper #1, in preparation) — replaces Green's-functions calibration and recovers the identifiable Carroll-6 subset per cell; (2) Track 2 — identifiability-limits map (paper #2, complete) — with prescribed transport, it asks which BGC closures real observations can constrain, and finds they can't sharply constrain any of the three (iron, calcite, growth): the binding constraint is the observing system, not the method. A full spatial emulator/UDE would be a forward/OSSE tool (which observations would break the limits), not an identifiability rescue — gated on direction, not built.

Installation

git clone https://github.com/2imi9/ECCO-DarwinDiff.git && cd ECCO-DarwinDiff
uv sync                     # create the environment from pyproject / uv.lock
uv run pytest -q            # smoke test (LLC270 / data tests self-skip)

Targets Python 3.11+ and PyTorch 2.4+. The synthetic demo needs nothing else; for real fits, point the loaders at the LLC270 tree via DARWIN_DATA_ROOT (and GLODAP_DATA_ROOT). Raw-data mechanics (Windows MAX_PATH, IC caches, per-loader layout) are in docs/cluster_setup.md and data/README.md.

Quick start

Run the synthetic recovery demo in ~5 min on a laptop or a free Colab T4 — notebooks/demo_colab.ipynb (). The core idea — a per-cell network whose loss backpropagates through the differentiable box — is a few lines of the public API:

import torch
from darwindiff.carroll6 import PARAM_BOUNDS, bounded_params, carroll6_integrate
from darwindiff.networks import DINN

env = torch.randn(3, 8, 16)                       # [SST, wind, MLD] over an 8x16 grid
dinn = DINN(n_input_channels=3, hidden_dim=16, n_outputs=6)
params = bounded_params(dinn(env), PARAM_BOUNDS)  # [6, 8, 16], bounded to physical ranges

state0 = torch.full((5, 8, 16), 0.1)              # [DFe, Ps, Pl, POC, PIC]
final = carroll6_integrate(state0, params, dt=0.25, n_steps=200)
loss = (final - target).pow(2).mean()             # target: an ECCO-Darwin v05 field
loss.backward()                                   # gradients through the whole simulation

For a real recovery on ECCO-Darwin v05 data, the runners are in scripts/ and every reported number is gated by the verified-experiment loop:

uv run python scripts/run_seasonal_recovery.py --aoi eqpac --n-seeds 10 --compile --out-dir runs/eqpac
uv run python scripts/verify_run.py runs/eqpac          # exit 0 == trustworthy

Headline result

Paper #1 (parameter learner). The iron pair (alpfe, scav_rat) recovers reproducibly — 38/40 (95 %) at the best 3-AOI config (~7 min/fit) — and the best config (geo1) holds {alpfe, scav_rat, R_PICPOC} jointly 7/10 seeds against real, Darwin-independent anchors (a 3-of-4-observable frontier); a per-cell-vs-global ablation confirms the per-cell network is load-bearing (7/10 vs 0/10). This is a consistency check against Carroll's own values, not a cross-validated discovery — the 0-D box homogenizes, so held-out real-data R² is negative. The honest target is 4 observable params: the growth pair {Smallgrow, Biggrow} is unobservable by construction, so "6/6" is the wrong frame, and R_PICPOC was never a wall (it recovers against real calcite; the deeper finding is that Carroll's global rain ratio is itself under-constrained and should be regional).

Paper #2 (identifiability-limits map). Adding prescribed transport does not turn the consistency check into a discovery: across the three targetable closures, real observations fail to constrain them for three distinct reasons — iron scav_rat = observability wall, calcite Ω-modulation = support-limited (the out-of-sample E2 is a decisive negative), growth = structurally unobservable. The binding constraint is the observing system, not the method — a map of what is observable is the contribution.

The full per-config record is the Config / Results Matrix; current best + known limits live in STATUS.md.

Documentation

📖 ecco-darwindiff.readthedocs.io

• Project status — canonical current-best snapshot + known limits
• Config / Results Matrix — what every config tested and found (single source of truth)
• DINN design · ECCO-Darwin relationship · Cluster setup · Data sources
• Archive — per-version research provenance (out of the onboarding path)

Repository layout

src/darwindiff/            Python package (importable as `darwindiff`)
  carroll6.py                5-tracer Carroll-6 box + Carroll's optima + bounds
  carbonate.py               Follows-2006 + Wanninkhof 2014 carbonate solver
  carroll6_5pft_2layer.py    2-layer 5-PFT integrator (seasonal + seed-batched)
  networks.py                DINN + DINNRegional + DINNDeep
  ecco_darwin_loader.py      Darwin v05 1° loader + AOI presets
  llc270_loader.py           native LLC270 monthly loader (xmitgcm)
  glodap_loader.py           GLODAPv2.2016b DIC/ALK loader
scripts/                   runners, overnight sweeps, verify_run.py, SLURM templates
notebooks/                 numbered notebooks 05–32; demo_colab.ipynb is the synthetic walkthrough
tests/                     pytest suite (runs in CI)
docs/                      results_matrix.md, dinn_design.md, cluster_setup.md, archive/

Citation

DarwinDiff is under active development; a formal manuscript and Zenodo DOI will be issued upon publication. In the interim, cite the repository directly:

@software{darwindiff_2026,
  author    = {{ECCO-DarwinDiff contributors}},
  title     = {{ECCO-DarwinDiff}: Differentiable Ocean Biogeochemistry
               for Per-Cell Parameter Recovery},
  year      = {2026},
  publisher = {GitHub},
  url       = {https://github.com/2imi9/ECCO-DarwinDiff}
}

Underlying ECCO-Darwin model + background reading (DOIs verified via OpenAlex)

If your work depends on the underlying model, cite Carroll et al. 2020 (JAMES) and 2022 (GBC).

Reference	Why it matters
Carroll et al. 2020 (JAMES)	Original ECCO-Darwin; the 6-parameter Green's-functions calibration we differentiate against.
Carroll et al. 2022 (GBC)	ECCO-Darwin v05; the publicly-accessible Darwin output is our active recovery target.
Dutkiewicz et al. 2009 (GBC)	Core Darwin biogeochemistry equations (carroll6.py).
Follows et al. 2006 · Wanninkhof 2014	Carbonate-system solver + air-sea CO₂ flux (carbonate.py).
Menemenlis et al. 2005 (MWR)	The Green's-functions calibration method DarwinDiff replaces.
Olsen et al. 2016 · Schlitzer et al. 2018	GLODAP DIC/ALK + GEOTRACES iron observations (loaders / losses).
Xu et al. 2025 (BINN)	Differentiable physics + per-location parameter network — closest method template.
Kochkov et al. 2024 (NeuralGCM) · Clark et al. 2026 (ACE2S) · Ouala & Lachkar 2026 (Neural-BGC)	Hybrid-physics / emulator references for Track 2.
Dheeshjith et al. 2024 (Samudra) · Yuan et al. 2026 (Samudra 2) · Ai2 2025 (SamudrACE)	AI ocean / coupled-climate emulators — architecture, resolution-scaling, and coupling templates for Track 2. SamudrACE names an explicit biogeochemistry hole as future work — the carbon-BGC-UDE slot Track 2 targets; none emulate ocean carbon (the whitespace). See ADR-0002.

Contributing

Contributions are welcome — read CONTRIBUTING.md first (scope-prefixed PR titles, commit/merge conventions, and the scripts/verify_run.py loop every reported number must pass). CI runs the test suite on every PR; run it locally with uv run pytest -q.

Acknowledgements

• MIT EAPS — research collaboration on ECCO-Darwin and the differentiable-physics approach.
• Northeastern Research Computing — the Explorer (H200) cluster.
• Massachusetts AI Compute Resource (AICR) / the Massachusetts AI Hub — B200 GPU access on the multi-institutional cluster (Boston University, Harvard, MIT, Northeastern, UMass, Yale).
• JPL ECCO Group + NASA NAS — ECCO-Darwin v05 outputs.
• GLODAP, GEOTRACES, NASA GHG Center — observational data products (active recovery targets).

License

Released under the MIT License — see LICENSE. The underlying ECCO-Darwin model is the work of the ECCO and Darwin teams and should be credited independently (citation block above).