Wave 2: datascope CSV mixed-date P1 fix + 2.4.0 bump - #7
Merged
Conversation
…face (P1) RE-AUDIT P1: the CSV loader coerced date-like strings to datetime on load, erasing the format before analyze_mixed_dates could see it — so a column mixing '2026-01-01' and '01/02/2026' was silently reconciled and never flagged, the exact silent coercion datascope exists to catch. Reproduced (0 findings on a mixed-format CSV), fixed (date-like CSV cells stay strings), regression-locked (test_dates_stay_strings_so_mixed_formats_can_surface). Excel date cells arrive typed from openpyxl and are unaffected. Also: bump 2.3.2 -> 2.4.0 (staged, not published); README missing-value threshold 40% -> 10% to match _DEFAULT_THRESHOLD_PCT; CHANGELOG entry. The published CLI still runs (now reports the mixed-date finding). Full suite 364 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prompt 6 step 0.e. core.hooksPath (the local pre-push guard) is inert on a fresh clone, so a force-added active engagement.yml could otherwise reach a deploy. Add an 'Engagement guard' step (python3 scripts/engagement_guard.py — python3 is preinstalled on ubuntu-latest) right after checkout in the deploy path, so the guard runs server-side regardless of local git config. No-op for demo/clean checkouts (engagement.yml is gitignored). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…v2.4.0 The README "Example Output" block was stale: it claimed 200 rows x 6 columns and 4 findings (2 Critical / 1 Warning / 1 Info) with a nonexistent `status` finding. The shipped sample_mixed_types.xlsx is 200 x 4 and the current CLI reports 2 findings, both Critical on revenue_mixed. Updated the block to the verbatim current run. Also regenerated the committed sample artifacts (HTML/PDF for both samples, annotated Excel for mixed_types) which were still emitted at v2.3.2 — refreshed to v2.4.0 (version string, print-CSS palette tokens, sales health-assessment copy). Finding counts unchanged (mixed 2, sales 5). Docs/artifacts only; no engine or version changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mestamp nondeterminism, regenerate samples
Conflict resolutions (Shawn-approved):
- datascope/__init__.py, pyproject.toml: version 2.4.0 (branch — newer release).
- CHANGELOG.md: union — [2.4.0]+[2.3.4]+[2.3.3]. Rewrote the branch's inaccurate
[2.4.0] "README threshold now reads 10%" line: 10% is _DEFAULT_THRESHOLD_PCT
(the flag floor), not the Warning split (severity.py: >=50). The line now
states both and notes it is a correction. Also noted the CSV-caveat removal.
- README.md: missing-value severity row now names both thresholds — "Flagged at
>=10% blank; Warning at >=50%, Info below" — the only phrasing that matches the
code (flag floor 10%, Warning split 50%). Removed main's stale CSV mixed-date
caveat ("supply as .xlsx"), which the branch's 2.4.0 loader fix contradicts.
- .github/workflows/publish.yml: synthesis — branch base (Engagement guard,
setup-python@v6, pypa trusted publishing) + main's workflow_dispatch trigger +
least-privilege permissions (top-level contents: read; id-token: write scoped
to the publish job). Narrowest scope wins on any permissions conflict.
Report byte-reproducibility:
- reports/html.py: the generator-tag timestamp honors SOURCE_DATE_EPOCH
(reproducible-builds standard); wall-clock otherwise. Same defect class Shawn
flagged in data-hygiene; datascope's renderer shared it.
- Regenerated samples/output/ with SOURCE_DATE_EPOCH pinned (2026-08-05). .html
and .pdf are byte-identical across processes; annotated .xlsx is
content-reproducible (openpyxl envelope is the documented residual).
- Added scripts/regenerate_samples.sh documenting the pinned epoch.
Verified: CSV mixed-date findings now surface (the 2.4.0 loader fix works on the
merged code); drift scan of regenerated artifacts = zero retired tokens;
tests 366 passed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 2 — datascope silent CSV date-coercion P1
P1 (RE-AUDIT): the CSV loader coerced date-like strings to
datetimeon load, erasing the format beforeanalyze_mixed_datesran — so a column mixing2026-01-01and01/02/2026was silently reconciled and never flagged. This is the brand-promise violation (datascope exists to surface silent coercion).datetime, datetime, str, datetime).test_dates_stay_strings_so_mixed_formats_can_surface— asserts str cell types + the mixed-date finding fires with the three detected formats.Also: version 2.3.2 → 2.4.0 (staged, not published — behavior change); README missing-value threshold 40% → 10% to match
_DEFAULT_THRESHOLD_PCT; CHANGELOG entry.Verification
🤖 Generated with Claude Code