Feature/second look UI#8
Open
HassanDawy wants to merge 8 commits into
Open
Conversation
The training tf.data pipeline feeds tf.image.decode_png(channels=1) tensors (shape (H,W,1)) into the numpy preprocessing helpers, which surfaced two crashes the first time modeling/train.py was run against a real build: 1. _to_grayscale rejected single-channel 3D arrays (only 2D / 3ch / 4ch were handled), raising "Unsupported image shape". A (H,W,1) array is already grayscale; squeeze the trailing axis. Fixed in both preprocessor.py and quality.py (the intentionally duplicated helper). 2. The module-global cv2.createCLAHE object is not thread-safe; under tf.data's num_parallel_calls=AUTOTUNE it crashed with an OpenCV ROI assertion / "Unknown C++ exception". Construct CLAHE per call instead. Safety behavior preserved: genuinely unsupported shapes still raise ValueError. Test suite remains green (33 passed, 2 skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
demos/preprocessing_grid.py renders the 8 stages of the preprocessor (raw -> grayscale -> CLAHE -> breast mask -> background removed -> pectoral removed -> orientation normalized -> 224x224 model input) on a real cached CBIS-DDSM scan and saves a slide-ready PNG. Defaults to a positive MLO case; --case <case_folder> selects another. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
demos/second_look_app.py: single-screen app demonstrating the on-device flow image -> preprocessing -> binary model -> confidence -> concern tier (Low / Moderate / Elevated via label_mapper). Pick a cached CBIS sample or upload an image; shows the 224x224 model input and a color-coded tier badge. Carries a prominent, hard-coded "placeholder model -- not yet trained" banner: the bundled checkpoint is a 1-epoch smoke model, so tiers are meaningless and for plumbing demonstration only. Adds gradio to requirements. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The app now leads with the binary classification the model actually outputs
("⚠️ Worth a second look" vs "✅ Not worth a second look", thresholded at 0.5
on P(WORTH_SECOND_LOOK)) as the primary result, with the Low/Moderate/Elevated
concern tier kept as supporting context. Also auto-selects the larger overnight
checkpoint when present, falling back to the smoke model. Placeholder banner
unchanged — predictions are still for plumbing demonstration only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
evaluate.py had never been run end-to-end; on a Windows cp1252 console the results printout crashed with UnicodeEncodeError on the "<-" arrow in the PRIMARY METRIC marker (and the header em-dash rendered as mojibake). Replaced both with ASCII so the evaluator runs cross-platform. Validated end-to-end: WORTH sensitivity 0.908 on the CBIS test split with the overnight baseline checkpoint; sensitivity-floor check prints PASS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolved data_pipeline/preprocessor.py and data_pipeline/quality.py by taking main's versions. Main's refactor (Dr. Fulton) already fixes both bugs this branch fixed, more thoroughly: - the (H,W,1) grayscale case is handled in the new data_pipeline/_imaging_utils .to_grayscale (shared by preprocessor + quality), and - _apply_clahe builds CLAHE per call (thread-safe under tf.data) and runs CLAHE on native bit depth before normalizing. So this branch's now-redundant edits to those two files are dropped. Also updated demos/preprocessing_grid.py to import to_grayscale/breast_mask from data_pipeline._imaging_utils, since main moved those helpers out of preprocessor.py. Grid regenerated; full test suite green (33 passed, 2 skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
demos/pipeline_diagram.py renders a factual DATA -> PREPROCESS -> MODEL -> EVALUATE/UX flow (each box labeled with its real source file) and saves a slide-ready PNG. Complements the image-level preprocessing_grid.png. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Brings main's preprocessor/quality refactor (_imaging_utils), the evaluate.py Windows fix, the grid import fix, and the pipeline diagram into the UI branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.