Skip to content

fix: count/nunique aggregates yield 0, not NaN, for parents without children#14

Open
CristobalSantana wants to merge 1 commit into
SantanderAI:mainfrom
CristobalSantana:fix/zero-count-aggregates
Open

fix: count/nunique aggregates yield 0, not NaN, for parents without children#14
CristobalSantana wants to merge 1 commit into
SantanderAI:mainfrom
CristobalSantana:fix/zero-count-aggregates

Conversation

@CristobalSantana

Copy link
Copy Markdown

What

In a one_to_many relation, aggregates are computed by grouping the child table and left-merging onto the parent. Parents with no child rows therefore receive NaN in every aggregate column, including count and nunique. Downstream, preprocessing bins those NaNs as missing.

The count of an empty set is 0, not a missing value. Leaving it as NaN conflates "no children exist" (a real, often highly informative value) with "data is missing", and the two become indistinguishable in the trained network. This felt like exactly the kind of hidden behavior the design principles ask contributors to avoid.

How I found it

I stress-tested the library with a synthetic public-tenders dataset (3,000 tenders, 8,000 bids, target = tender declared void). 287 tenders had zero bids, which was by construction the strongest predictor of the target (80% positive rate vs 22% global). After materialization, all 287 had bid_count = NaN and were binned as missing. With the fix, on that dataset, ROC-AUC went from 0.753 to 0.808 and PR-AUC from 0.533 to 0.608. The metric gain is dataset-specific, but the semantic issue is general: in real data that also has genuinely missing child records, the two cases cannot be separated at all without this distinction.

Change

After the aggregate merge, fill NaN with 0 only for columns produced by count and nunique ops. Other ops (mean, latest, etc.) still yield NaN, since they are truly undefined over an empty set. This also covers windowed counts (window_days), where a parent with children outside the window now gets 0 instead of NaN.

Note this is a behavior change for anyone relying on NaN counts; given 0.1.0 alpha status I assumed the semantic fix is preferred, but happy to gate it behind a config flag if you'd rather keep the current default.

Testing

New regression test: parent with zero children gets count == 0, nunique == 0, and mean stays NaN. uv run ruff check . clean, full test suite passes (63 tests), smoke test auto-bayesian train examples/lead_scoring.toml runs.

@CristobalSantana CristobalSantana requested a review from a team as a code owner July 4, 2026 21:02
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@CristobalSantana

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@CristobalSantana CristobalSantana force-pushed the fix/zero-count-aggregates branch from abafc91 to 474a604 Compare July 4, 2026 21:26
github-actions Bot added a commit that referenced this pull request Jul 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant