Skip to content

Port ddrs ddr_match corrections: outflow_idx fix, neg-solve counter, signed tau, β + Cunge X - #192

Merged
taddyb merged 9 commits into
DeepGroundwater:masterfrom
taddyb:port-ddrs-corrections
Aug 18, 2026
Merged

Port ddrs ddr_match corrections: outflow_idx fix, neg-solve counter, signed tau, β + Cunge X#192
taddyb merged 9 commits into
DeepGroundwater:masterfrom
taddyb:port-ddrs-corrections

Conversation

@taddyb

@taddyb taddyb commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Ports the corrections the ddrs repo developed behind its ddr_match config flag (where ddr_match: true reproduced this repo bit-for-bit and false enabled corrected behavior — all ddrs production configs now run false). This lands as a clean break: no compatibility flag; old results reproduce from old commit hashes.

Sources of truth: ddrs .claude/PHYSICS-CORRECTIONS.md, docs/2026-08-03-ddr-match-findings.md, docs/2026-08-06-tau-sweep-pilot-findings.md §5i. Spec and plan are committed under docs/superpowers/.

Changes (one commit per stage, in dependency order)

  1. fix(merit): gauge-reach outflow_idx extraction — the MERIT collate summed a gauge's upstream columns, dropping the gauge reach's own local drainage (0.215× observed on the worst small basins; 26/1841 gauges below 0.5× baseline). The MC solve at the gauge reach already carries upstream flow plus its own lateral inflow. Side effect: partial-area flow_scale corrections now actually reach MERIT predictions.
  2. feat(routing): negative-solve counter — counts solve output < 0 before the discharge_lb clamp rewrites it (mass creation), logs the per-forward rate. Tripwire for stage 4 (~0.1% expected, ddrs-measured).
  3. feat(tau)!: signed-at-zero tau convention, default 9 — slice [tau : -(24−tau)], pooled day i scored against obs day i (tau=0 day-aligned). The shipped legacy tau=3 was signed −8, mis-set in the wrong direction; ddrs measured the fix alone at +0.086 median NSE, flipping routing from losing to the summed-Q' baseline to beating it. Also unifies train.py's inconsistent inline slice ([13 : -11+tau], a fractional-day window) with the shared helper. Legacy pin: new tau=16 cuts the same hour window as legacy tau=3.
  4. feat(routing)!: trapezoid-exact celerity + Cunge-derived Xc = v·β with β = 5/3 − (4/3)·A·√(1+z²)/(T·P) (the hardcoded 5/3 is the wide-rectangular limit, 22–27% high on real channels), and X = clamp(0.5·(1 − Q/(T·S·c·L)), 0, 0.5) computed per timestep, replacing the static per-reach x (MERIT constant 0.3, Lynker zarr) whose plumbing is removed end to end. Autograd supplies the backward.

Deliberately not ported (measured NO-GOs in ddrs): enforce_positivity, reach subdivision, Courant sub-stepping, slope-floor removal, leakance.

Breaking changes

  • All pre-port checkpoints are invalidated (trained on legacy physics and the legacy tau window); config tau values do not carry across the convention change.
  • RoutingDataclass.x is removed.

Known trade

Cunge X ≈ 0.5 on most CONUS reaches narrows the non-negative-coefficient window 2X ≤ Cr ≤ 2(1−X); negative Muskingum coefficients rise (artifacts), while negative solves stayed ~0.1% in ddrs. The stage-2 counter reports the rate every forward — a retrain showing a rate far above ~0.1% is a stop-and-investigate signal.

Test plan

  • Full unit suite: 643 passed, 1 skipped
  • New gates: β limits + finite-difference dQ/dA (1e-6), Cunge diffusion-matching identity, c1+c2+c3 = 1, gradient flow through the X path, tau legacy-equivalence pin, gauge-reach extraction
  • User-run: MERIT retrain — expect routing to beat the summed-Q' baseline directionally (ddrs: 0.706 vs 0.642 median NSE); retrains at the stage-3 vs stage-4 commits attribute tau vs physics separately

🤖 Generated with Claude Code

taddyb and others added 9 commits August 2, 2026 11:17
…LID)

Generated by ~/projects/ddrs/scripts/build_gages_2000_area_balanced.py
(seed 42). Corrects the absolute DA_VALID criterion that preferentially
deleted large basins; see gage_info README for the derivation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…counter, signed tau, β + Cunge X

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ported from ddrs (.claude/PHYSICS-CORRECTIONS.md): summing upstream columns
dropped the gauge reach's own local drainage (0.215x observed on the worst
small basins; 26/1841 gauges below 0.5x baseline). The MC solve at the gauge
reach already carries upstream flow plus its own lateral inflow. Side effect:
partial-area flow_scale corrections now actually reach MERIT gauge
predictions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The clamp to discharge_lb silently creates mass and hides Courant
instability. ddrs measured ~0.03-0.14% of reach-timesteps negative under
corrected physics; this counter is the tripwire for the Cunge-X port.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Slice [tau : -(24-tau)], pooled day i scored against obs day i (dMC-Juniata
sign; tau=0 day-aligned). Shipped legacy tau=3 was signed -8 — mis-set in the
wrong direction; ddrs measured the fix alone at +0.086 median NSE, flipping
routing from losing to the summed-Q' baseline to beating it. Also unifies
train.py's inline slice (was [13 : -11+tau], a fractional-day window) with
the shared helper. BREAKING: pre-port checkpoints trained on the legacy
window; config tau values do not carry across.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
c = v·β with β = 5/3 − (4/3)·A·√(1+z²)/(T·P) — the hardcoded 5/3 is the
wide-rectangular limit, 22-27% high on real channels. X is now computed per
timestep from Cunge's diffusion matching, clamp(0.5·(1 − Q/(T·S·c·L)), 0,
0.5), replacing the static per-reach x (MERIT constant 0.3, Lynker zarr) —
that plumbing is removed. Ported from ddrs ddr_match:false physics
(.claude/PHYSICS-CORRECTIONS.md). BREAKING: pre-port checkpoints were
trained against the legacy physics. Known trade: Cunge X≈0.5 narrows the
non-negative-coefficient window; the negative-solve counter reports the
clamp rate every forward (~0.1% expected, ddrs-measured).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dits

Master (DeepGroundwater#188) replaced the static architecture doc with the LLM-wiki
schema; the tau/celerity/Muskingum-x corrections this branch made to the
old doc now live in the local wiki/ pages instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI resolves ruff 0.16.3, which now formats Python code blocks inside
markdown; local lock has 0.15.4, which does not. Applied 0.16.3's
formatting so the ruff CI job passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@taddyb
taddyb merged commit 3125dac into DeepGroundwater:master Aug 18, 2026
4 checks passed
@taddyb
taddyb deleted the port-ddrs-corrections branch August 18, 2026 02:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant