feat(cadecon): noise-constrained sparsity for low-SNR deconvolution

[daharoni]

Summary

Adds an optional, knob-free sparsity to the InDeCa spike inference that suppresses noise being fit as spurious activity. Rather than picking the binarization threshold that maximizes fit (which overfits into the noise floor), it picks the sparsest spike support whose reconstruction residual still reaches the data-derived noise floor — "don't spend spikes explaining anything that's within the noise." The effect self-scales with SNR: strong at low SNR, negligible at high SNR.

Default ON in the CaDecon app (library default is off / MaxPve, so existing callers are byte-identical). There is no tuning parameter — σ is measured from the data.

How it works

• Threshold-selection stage, not FISTA. The Box[0,1] relaxation is unchanged (an L1 λ there gets laundered out by the peak-normalization that follows). The sparsity acts only when the relaxed solution is binarized: pick the highest (sparsest) threshold whose residual, evaluated at original-rate grid positions, stays within the noise budget σ²·n_grid (the Morozov discrepancy principle). Falls back to the min-residual threshold if none is feasible.
• LP-robust noise estimate. σ is the raw high-frequency-PSD estimate of the unfiltered trace (immune to the low-pass filter, which would otherwise gut that band) rescaled by the empirically measured noise gain of the actual upsample→HP/LP chain, so the budget tracks the noise that survives into the residual.
• Consistent outer-loop selection. The scale-iteration loop selects its best iterate by max-PVE under MaxPve (unchanged) but by best prescale calibration under NoiseFloor, so the outer loop doesn't re-select the densest-fitting iteration and undo the sparsity the inner search just imposed.

Teaching / impact overlay (opt-in, default OFF)

To see the sparsity's effect, an opt-in overlay solves each trace with both settings per iteration and stores the opposite-setting counts; the Trace Inspector overlays them and shows both spike counts. It's precomputed during the run (no main-thread solving) so browsing stays smooth, roughly doubles run time, and lives under a Display Options panel — a visual-inspection aid, not an algorithm knob. Documented in-UI and in the "Reading Convergence & Results" tutorial.

Also in this PR

• Removed the experimental collapse_runs lever — it collapsed genuine dense spike trains and was shown harmful on bursty data; only the noise-constrained path ships.
• Python binding + docs. calab.solve_trace(..., noise_constrained=False) exposes the option through PyO3; documented in the CaDecon guide.
• Threads the flag through the WASM + PyO3 bindings.
• Sidebar layout fix: control panels used the default flex-shrink, so a short viewport (or large text) compressed and overlapped them instead of scrolling. Panels are now fixed-size and the sidebar scrolls.
• Changelog backfill: reconstructed the missing minor releases (2.1.0–2.4.0), added 2.5.0 (PRs fix(cadecon): correct per-subset kernel attribution + init/variance robustness #153–docs(changelog): backfill the #60–155 gap from git history #167) and 2.6.0 (this PR) from git history.

Testing

• Rust: cargo test -p calab-solver — 135 passed. New tests cover the noise-floor budget-ordering invariant (larger budget → sparser support), the infeasible-fallback path, high_band_sigma recovering an injected white-noise σ, and end-to-end recovery on a noisy trace.
• Python: pytest — 18 passed (adds a noisy-trace noise_constrained integration test).
• tsc clean; CI green (rust, python 3.11/3.12/3.13, format, supabase).
• Event-level A/B validation (simulated cells, event precision/recall + count-bias vs ground truth across SNR ∈ {1,2,4,8,16}): noise-constrained more than doubles precision and pulls count-inflation from ~2.8× toward ~1× at every SNR. By F1 it wins at SNR 4/8, is a wash at SNR 2 (a precision-for-recall rebalance), and is mildly worse at SNR 1 (the documented low-SNR trim — the toggle description flags it). A ±30% kernel-mismatch condition barely changed the results. Note: smoothed-trace correlation is a misleading gauge here (it rises with the feature even where event recall falls), which is why validation is event-level.
• Manual: verified in-browser (demo data) that noise-constrained sparsity cleans low-SNR activity, the overlay + per-cell spike counts render with no lag, and the sidebar scrolls without squishing at short viewport heights.

🤖 Generated with Claude Code