NLP-Melanoma

Code and application for the manuscript “Development and validation of NLP models for automated melanoma service-level monitoring.”

This repository is organised around two deliverables:

1. Model-development code: reusable Python modules and scripts used to preprocess data, train/evaluate task-specific models, run public transformer comparators, and generate manuscript tables.
2. Melanoma monitoring app: a Streamlit application for local retrospective processing of pathology reports, denominator filtering, institutional PDF parsing, and structured output generation.

The pipeline is intended for retrospective service-level monitoring and local validation. It is not intended for point-of-care diagnosis, clinical decision-making, or replacement of expert pathological review.

Citation

If you use, adapt, redistribute, or build upon this code, model artifacts, or application in academic work, service evaluation, derivative software, reports, presentations, or other outputs, please cite the associated manuscript and this repository.

GitHub should display the preferred citation through the Cite this repository button, generated from CITATION.cff. Additional attribution information is provided in NOTICE and docs/CITATION_AND_LICENSE.md.

---

Quickstart for reviewers

Recommended Python version: 3.11. The app requirements pin scikit-learn 1.4.2 to match the saved model artifacts.

py -3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install -r melanoma_app\requirements.txt
python scripts\smoke_test_app.py
cd melanoma_app
python -m streamlit run app.py

See docs/QUICKSTART_REVIEWERS.md for a fuller step-by-step guide.

App release asset

For most users, the easiest entry point is the packaged app release asset:

melanoma_monitoring_app_v1.0.0_with_models.zip

The repository contains the source code. The release asset contains the app folder, selected model/vectorizer artifacts and configuration, templates, and synthetic demo data in a single downloadable package.

Important documentation

• docs/QUICKSTART_REVIEWERS.md: install, smoke test, and run instructions.
• docs/DATA_AVAILABILITY.md: privacy and data-sharing statement.
• docs/APP_USER_GUIDE.md: app workflow, denominator, warnings, and multilingual interface.
• docs/RELEASE_NOTES_v1.0.0.md: release packaging notes and checklist.
• docs/CITATION_AND_LICENSE.md: citation, attribution, and license information.
• docs/model_cards/: short model cards for A, B, C1, and C2.

Repository structure

NLP-Melanoma-main/
├── src/                         # New reusable pipeline code
│   ├── core/                    # preprocessing, vectorisation, training, evaluation, bootstrap, e2e
│   ├── rule_based/              # deterministic rule-based comparators
│   └── sota/                    # public transformer comparator code
├── scripts/                     # New command-line scripts for benchmarks/tables/preprocessing
│   ├── preprocessing/           # denominator-filter script
│   ├── make_table2_development_selection.py
│   ├── make_table3_module_specific.py
│   ├── make_table4_e2e.py
│   ├── run_sota_binary_zero_shot.py
│   └── run_sota_c2_qa.py
├── artifacts/                   # Selected model artifacts + aggregate outputs, no raw clinical text
├── reports/                     # Manuscript-ready aggregate tables/results
├── melanoma_app/                # App packaged with the selected models/configuration
├── docs/                        # Pipeline, SOTA, app, packaging and validation notes
├── examples/                    # Synthetic examples only
├── data/                        # Placeholder only; raw clinical data are not included
├── requirements.txt
└── requirements_sota.txt

This version intentionally does not include the old exploratory notebooks as the primary code. The development workflow is represented by modular code in src/ and executable scripts in scripts/.

---

Manuscript-aligned selected modules

A   Melanoma presence                         SVC with SMOTE
B   Invasiveness among melanoma               Random Forest
C1  Breslow availability/routing/completeness Random Forest with SMOTE
C2  Numeric Breslow extraction                Catalan RoBERTa QA ensemble

End-to-end routing:

All reports → A
Predicted melanoma → B
Predicted invasive melanoma → C1
Predicted Breslow available → C2

C1 is an intermediate routing/completeness step. Operational Breslow benchmarking should use numeric extraction coverage and accuracy. Breslow-based T-stage outputs are proxies only because ulceration is not extracted.

---

Install for model-development code

Recommended environment: Python 3.11.

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

For public transformer comparators:

python -m pip install -r requirements_sota.txt

---

Run manuscript table generation

The scripts assume the expected project layout and locally available approved input data or existing aggregate artifacts.

python scripts/make_table2_development_selection.py
python scripts/make_table3_module_specific.py
python scripts/make_table4_e2e.py

Public transformer comparator scripts are available in scripts/run_sota_*.py.

---

Run the app

The app is in melanoma_app/ and includes the selected model/vectorizer artifacts and configuration used by the deployable pipeline.

On Windows:

melanoma_app/run_melanoma_analyzer.bat

Or manually:

cd melanoma_app
python -m pip install -r requirements.txt
python -m streamlit run app.py

The app includes:

• lesion/sample-level parsing for A/B/C-labelled reports,
• institutional PDF presets for Vall d’Hebron and Germans Trias i Pujol,
• editable denominator/suspicion filtering,
• end-to-end A → B → C1 → C2 routing,
• CSV export with audit columns.

---

Data availability and privacy

No clinical free-text reports, PDFs, or patient-level datasets are included.

The folders data/raw/ and data/cache/ contain README placeholders only. Users must provide locally approved, appropriately anonymised data to reproduce model development.

Model artifacts are included to support reproducibility and local validation. Before making modified artifacts public, run:

python scripts/audit_model_artifacts.py melanoma_app/models

This lightweight check is not a substitute for formal privacy review.

---

Suggested GitHub release assets

For usability, publish the repository code normally and attach the app as a release asset, for example:

melanoma_monitoring_app_v1.0.0_with_models.zip

The app folder in this repository already contains the model/vectorizer artifacts and configuration needed for the manuscript-aligned pipeline, except for external public transformer weights that may be downloaded by Hugging Face when required.

---

Synthetic examples

Synthetic examples are provided under examples/synthetic_reports/ and can be regenerated with:

python scripts/create_synthetic_examples.py

These reports are invented from scratch. They preserve only structural patterns needed for testing: section headers, A/B lesion/sample labels, administrative footers, dates, positive melanoma, benign melanocytic lesions, non-melanocytic exclusions, and multi-lesion reports. Do not create examples by lightly editing real reports.

App interface update

The Streamlit app includes a guided workflow (upload → cohort denominator → NLP pipeline → results/export), a denominator confirmation step, and a multilingual interface in Catalan, Spanish, and English. Technical CSV output column names remain stable in English.