populace-fit: weight-aware conditional models (regime-gated chained QRF)

[MaxGhenis]

The conditional-models operator — populace.fit, the second shard of the stack (after populace-frame). Per DESIGN.md ("populace-fit: conditional models").

What's here

• ConditionalModel / FittedModel protocols (model.py) — fit(frame, predictors, targets, *, weights="design") -> FittedModel; FittedModel.predict(frame_or_df) -> DataFrame (one draw column per target). Weight-aware by construction: weights selects which typed weight vector of the owning entity to use (default that entity's design weights), reading the Frame's typed Weights rather than a raw array. weights="none" is the only way to fit unweighted, and a misspelled or mismatched kind raises (naming the culprit) instead of silently falling back to unweighted — closing the 2026-06 microimpute landmine at the type boundary. resolve_fit_weights is the single authority for this rule.
• QRF / RegimeGatedQRF (qrf.py) — the canonical model: regime-gated (structural, unweighted sign-mixture gates), sequentially chained (each target conditions on the predictors plus the targets already drawn), quantile-regression-forest draws (a seeded per-row quantile), with the frame's weights materialized by weighted bootstrap (importance-resample the training rows by weight before growing each forest). Reimplemented from scratch against the Frame — it does not import microimpute. This is the microimpute#196 fix as the reference mechanism.
• Import-time kernel-compat assert (__init__.py) — the charter's constellation-versioning mechanism: populace-frame's series is checked at import (pre-1.0, the 0.x minor; major from 1.0 on), so a loose resolver that ignores [tool.uv.sources] cannot silently assemble an incompatible kernel pair.

The headline contract

test_weighted_fit_shifts_draws_toward_weighted_truth is the real realization of the placeholder the kernel left skipped in packages/populace-frame/tests/test_contracts.py. On a donor whose target is large exactly where weight is small (the #196 shape; the high-value regime is independent of the predictors, so honoring the weight is the only way to recover the weighted conditional):

• (a) the weighted fit's draws' mean lands within 20% of the true weighted mean;
• (b) weights="none" lands within 20% of the unweighted mean;
• (c) the two differ by >3x.

This is the microimpute#196 bug class — now a standing guarantee of the stack rather than a latent footgun. Follow-up: the kernel's skipped test_weighted_fit_shifts_draws_toward_weighted_truth placeholder can be unskipped/removed once this shard is in the workspace (it could not be edited from this branch's scope).

Plus: regime gates preserve a zero-inflated target's zero mass and both signs (no zero-crossing); chaining reproduces a cross-target correlation; predict row-count/index match the input; fixed seed is deterministic; weights="none" is the only unweighted path. 35 tests, n=5000 seeded for CI speed.

Note on the scikit-learn pin

scikit-learn is capped >=1.5,<1.9. scikit-learn 1.9 removed sklearn.tree._tree.DTYPE, which quantile-forest imports — so an unbounded >=1.5 resolves to 1.9 and import quantile_forest fails. On the workspace's Python 3.14 interpreter the cap keeps the only working combination (scikit-learn 1.8 + quantile-forest 1.4) resolvable; the cap can be lifted once quantile-forest tracks the 1.9 tree ABI.

Optional heavy deps (scikit-learn, quantile-forest) stay in this shard, never in populace-frame.

Validation

uv sync --all-packages && uv run pytest packages/populace-fit && uv run ruff check packages/populace-fit — all green (35 passed; ruff clean). Full workspace suite: 192 passed, 3 skipped (the microunit/policyengine_us-gated kernel tests).

🤖 Generated with Claude Code

[MaxGhenis]

Adversarial review — converting to draft; soundness fixes needed before merge

An independent clean-room review (every finding reproduced) found bugs in the operator's core purpose. Marking draft. Ranked:

HIGH — the weighted-bootstrap gate annihilates rare low-weight classes. The gate is fit on an n-of-n weighted resample, so a scarce-but-oversampled stratum (exactly what the charter's pool design carries: oversampled in rows, downweighted in mass) gets zero rows in the resample → single-class gate → that class drawn with probability 0, forever. Reproduced: 9/10 gates single-class, 0 positive draws in 2M rows vs ~80 expected. The fit re-rarefies what the pool deliberately oversampled — the opposite of "tail support is strata's job." Fix: HistGradientBoostingClassifier honors sample_weight exactly; fit the gate weighted, drop its bootstrap (the forests still need the bootstrap — QRF ignores weight magnitude, confirmed at _quantile_forest.py:266).

HIGH — household-weights-only frames can't be fit weighted at all. The canonical CPS shape (person-level fit, household design weights) fails weights_for("person"); only weights="none" runs. The operator built to prevent unweighted-by-accident makes the unweighted escape hatch the only thing that runs on the most representative input — microimpute#196's social mechanism, rebuilt. Fix: resolve through effective (broadcast) weights — touches kernel API (_effective_weights is private; accounting already uses it, so wmean is weighted on a frame where fit refuses to be).

HIGH — NaN targets silently become zeros in gated regimes (NaN-blind sign labels → zero class). Survey item-nonresponse silently moves mass to $0. Fix: validate finite at fit, raise naming the column + count.

MEDIUM — tail mass undershot ~2x, and the contract test passes for the wrong reason ("delete all low-weight rows" also passes — it asserts only means, never that the rare regime survives). Plus the 201-point grid winsorizes draws to [0.5%, 99.5%]. This is the capital-gains/dividends tail problem at the method level. Fix: assert high-draw-share survival, draw leaf values as atoms incl. endpoints.

MEDIUM — fit and draw RNG share one stream (predict quantiles bit-identical to the gate's bootstrap uniforms; max|diff|=0). Fix: SeedSequence(seed).spawn(2).

LOW — missing populace-frame>=0.1,<0.2 pin (charter mandate; gate works but resolution-time failure is the better failure); impossible-remediation error message; duplicate-target / target-in-predictors unguarded; (n×201) memory at design scale.

Sound, must not regress: the weighted factorization P(sign|x)·P(y|x,sign); chained-equations semantics (drawn values fed forward); resolve_fit_weights as the single enforcement point; the compat gate; determinism. The headline contract does kill the literal #196 bug — it just can't see tail mass, rare-class survival, or the household-weighted frame it never builds.