Fix lag chart x-axis labels, add driver correlation/VIF confounding check#17
Merged
Conversation
The lag_weeks ordinal axis defaulted to a steep label angle, rendering 0-8 sideways instead of upright.
Per-driver findings were always univariate (each driver tested against hospital_demand alone) with no visibility into whether the drivers are correlated with each other. Adds driver_correlation_matrix() and driver_vif() to the analysis layer, a live read-only "Driver correlation matrix (confounding check)" expander in the dashboard's Correlation & Regression tab, and a one-paragraph README caveat pointing there instead of baking specific numbers into static text.
The bundled weather sample spans a fixed ~1-year window frozen at make_samples generation time, but run.py (and the fallback tests) request a window relative to today. Once the committed sample aged past the trailing-365-day window, _load_sample clipped to zero rows — the sample tier silently dropped the signal. That broke the daily-ingestion workflow's gating test job (needs: test) every scheduled run since 2026-06-26, blocking real ingestion even though live Open-Meteo works. Make the fallback genuinely fail-soft: slide the sample by whole years until it covers the requested window before clipping. The planted signal is day-of-year seasonal, so a whole-year shift preserves it; shifting the naive timestamps before localizing keeps the existing DST guards in charge of the spring-forward/fall-back hours. The same latent clip-to-empty behavior exists in wastewater.py and hospital.py sample loaders (not addressed here — their tests don't use a today-relative window, so CI isn't failing on them). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011FPpSio8V9rxWMCG5RnYxB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
driver_correlation_matrix()(src/analysis/correlate.py) anddriver_vif()(src/analysis/regression.py) — per-driver findings elsewhere in this app are univariate (each driver tested againsthospital_demandalone), with no visibility into whether the drivers are correlated with each other, which can make an independent-looking effect actually reflect a correlated peer.Test plan
pytest tests/ -q— 103 passed (8 new tests coveringdriver_correlation_matrixanddriver_vif)streamlit run src/dashboard/app.pywith a headless browser: lag chart labels render upright; new expander renders a correct heatmap + VIF table, independently cross-checked against a standalone re-derivation of the same numbers from the raw parquet datahttps://claude.ai/code/session_011FPpSio8V9rxWMCG5RnYxB
Generated by Claude Code