Skip to content

test: cover riskiest untested production paths (PII filter, pruner, judge, conversation builder, ingestion pipeline, preference formatter) - #37

Merged
omarjson merged 2 commits into
mainfrom
hoplite/messene-ithome-b24246a8
Sep 2, 2026
Merged

test: cover riskiest untested production paths (PII filter, pruner, judge, conversation builder, ingestion pipeline, preference formatter)#37
omarjson merged 2 commits into
mainfrom
hoplite/messene-ithome-b24246a8

Conversation

@usehoplite

@usehoplite usehoplite Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds 94 focused unit tests for the six least-covered, highest-risk production modules (all below 32% coverage before; 94–98% after). These paths gate security (PII/secret redaction), training-data quality (pruning), and user-visible output correctness (conversation generation, DPO export), so their behavior is now pinned down.

Module Coverage before Coverage after
core/pii_filter.py 0% 98%
synthesis/pruner.py 19% 94%
synthesis/conversation_builder.py 0% 97%
ingestion/pipeline.py 22% 97%
synthesis/judge.py 31% 97%
exporter/formatters/preference.py 22% 94%

Production bug found and fixed

distill_align.core.pii_filter crashed on import: the BEARER_TOKEN regex embedded the inline (?i) global flag mid-pattern ((?:(?i)...), which Python's re rejects with re.error. Since ingestion/auto.py lazily imports PIIFilter when scan_pii is enabled, the entire PII-scanning feature was dead on arrival. The flag was moved to the start of the pattern; semantics unchanged. This was the first thing the new PII filter tests caught.

Two existing behaviors are documented in tests rather than silently changed: consecutive-digit credit-card numbers are classified as phone_us (pattern ordering shadows credit_card — noted in test_plain_card_number_shadowed_by_phone), and the quality gate does not reject conversations for short content or invalid structure alone (scores 0.7/0.6 respectively, still above the 0.5 threshold).

Verification

  • Full unit + property suite: 297 passed; unit + integration: 292 passed
  • Targeted modules: coverage verified via pytest --cov
  • ruff check clean on all changed files
  • Total repo coverage: 43% → 50%

Open in Hoplite

usehoplite Bot and others added 2 commits September 2, 2026 20:59
…n pipeline, preference formatter

Adds 94 focused unit tests for the least-covered production paths (0-31%
before, 94-98% after) and fixes a latent import-time crash in the PII
filter: the BEARER_TOKEN pattern embedded the inline (?i) flag mid-pattern,
which Python's re rejects, so PIIFilter (and the scan_pii ingestion path)
could never be imported.

Co-authored-by: Omar <omr.omr163@gmail.com>
Runs ruff format (0.9.10, matching the CI-pinned version) on the four new
test files and adds an [Unreleased] CHANGELOG section covering the new
tests and the PII filter import fix, satisfying the changelog-enforcer.

Co-authored-by: Omar <omr.omr163@gmail.com>
@usehoplite

usehoplite Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Fix pass for the three failing checks:

  • Lint (Ruff)ruff check passed but ruff format --check failed on four new test files. Reformatted them with the CI-pinned ruff 0.9.10.
  • changelog — Added an [Unreleased] section to CHANGELOG.md covering the new tests and the PII filter fix.
  • Dependency Review — Cannot be fixed from code. The log says: "Dependency review is not supported on this repository. Please ensure that Dependency graph is enabled." This is a repository setting: the GitHub Dependency graph feature (Settings → Code security and analysis) must be enabled for actions/dependency-review-action@v4 to run. That requires owner/admin access; once enabled, the check should pass on the next run (this PR changes no dependencies).

@omarjson
omarjson merged commit 425ad3c into main Sep 2, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant