sedaudit

Audit your SED fits before reporting them: detect prior-dominated parameters and quantify information gain.

sedaudit computes D_KL(posterior || prior), chi²/dof, and quality flags from galaxy SED-fitting posteriors. It answers the question: did the data actually constrain this parameter, or is the posterior just the prior?

This matters because SED fitting pipelines can silently produce meaningless results — posterior samples that are just draws from the prior, with no information gained from the data. Without checking, you might report "stellar mass = 10^9.0 M_sun" when the data only told you "somewhere between 10^6 and 10^12."

Quick start

import sedaudit
import numpy as np

# Your posterior samples: shape (n_samples, n_params)
samples = np.load("posterior_samples.npy")

# The prior bounds you used in the fit
prior_bounds = {
    "massformed": (6.0, 12.0),
    "age": (0.01, 5.0),
    "tau": (0.01, 3.0),
    "metallicity": (0.005, 5.0),
    "Av": (0.0, 5.0),
}

# Run the audit
report = sedaudit.generate_report(samples, prior_bounds)

# Print human-readable summary
print(sedaudit.format_report_text(report))

Output:

============================================================
SED AUDIT REPORT — Object unknown
============================================================

Parameter          Median     D_KL        R           Status
------------------------------------------------------------
massformed        10.1335   3.8129   0.0074               OK
age                0.0211   4.0728   0.0007             EDGE
tau                1.5573   0.0780   0.6179        PRIOR-DOM
metallicity        1.0117   2.6845   0.0209               OK
Av                 2.3560   3.1215   0.0227               OK

Fit quality: CAUTION
  Constrained: 4, Prior-dominated: 1, Edge-clustered: 1
============================================================

• D_KL >> 0.5: the data constrained this parameter. You can report it.
• D_KL ≈ 0: prior-dominated. The posterior just reflects your prior, not the data.
• R (width ratio): (84th - 16th percentile) / (prior range). R ≈ 1 = prior, R << 1 = constrained.
• EDGE: posterior clustering near a prior bound. This is a signal to investigate, not necessarily a problem — a genuinely young galaxy with high D_KL can legitimately cluster at the low-age edge. But if EDGE appears with low D_KL, the prior may be cutting off the true posterior.

Reading from Bagpipes

from sedaudit.io import load_bagpipes_posterior, BAGPIPES_DELAYED_COLUMNS

data = load_bagpipes_posterior("pipes/posterior/my_run/12345.h5")
report = sedaudit.generate_report(
    data["samples"],
    prior_bounds={"age": (0.01, 5.0), "massformed": (6, 12),
                  "metallicity": (0.005, 5.0), "tau": (0.01, 3.0), "Av": (0, 5)},
    param_names=BAGPIPES_DELAYED_COLUMNS,
    lnlike=data["lnlike"], n_bands=6, n_params=5,
    object_id="12345",
)

What it checks

Check	What it catches
D_KL < 0.5	Prior-dominated parameters — the data didn't constrain this
chi²/dof >> 1	Model can't fit the data (bad model, not bad data)
Flat likelihood	Sampler didn't explore — pipeline bug
Edge clustering	Posterior hitting prior bounds — investigate (may need wider priors, or may be genuine)

Notebooks

Two example notebooks with real Bagpipes H5 posteriors from JADES DR5 (included in examples/data/):

• bug_detection.ipynb — The broken fit that motivated this tool. Shows a Bagpipes run where every posterior was just the prior, and how sedaudit catches it immediately.
• interpreting_results.ipynb — How to read sedaudit reports for fits that are not broken. Five real galaxies showing OK, EDGE, weakly constrained, and borderline cases.

Data: JADES DR5 NIRCam photometry, objects 2532, 165902, 1027544, 439876, 446253, 185898.

The bug that motivated this tool

While fitting 585 Little Red Dots from JADES DR5 with Bagpipes, I used the model key 'sfh' instead of 'delayed'. Bagpipes 1.3.5 silently ignored the SFH component, producing zero-flux models. The fits completed without errors — but every single "posterior" was just the prior. D_KL was ≈ 0 for all parameters. I almost published these results.

sedaudit would have caught this on the very first fit: D_KL ≈ 0 everywhere, chi²/dof in the thousands, and a flat likelihood with only 1 unique value.

Limitations

• Currently only reads Bagpipes H5 posteriors directly. Prospector and CIGALE support planned.
• Assumes uniform (flat) priors for D_KL computation. Gaussian priors require a different formula.
• D_KL is estimated via histograms (not KDE), which works well for n_samples > 500.
• Does not support hierarchical or joint fits — one object at a time.
• D_KL quantifies information gain relative to the prior; it does not validate whether the model itself is appropriate for the data. A model that cannot reproduce the observed SED (high chi2/dof) but constrains parameters far from prior bounds will still show high D_KL. Always interpret D_KL alongside chi2/dof.

Installation

pip install sedaudit

For plotting support:

pip install sedaudit[plot]

Reference

The D_KL framework for SED fitting is described in:

• Buchner, J. (2022). "An intuition for physicists: information gain from experiments." RNAAS, 6, 124. doi:10.3847/2515-5172/ac6b40 arXiv:2205.00009

License

MIT