Maturity

Field	Value
Status	Research prototype
Maturity	Low
Field-ready	No — not clinical, not field-certified
Last reviewed	2026-08-07

Archived NIH ChestX-ray14 training notebook. Research only — not for clinical use.

Archived — superseded by later work. Preserved for reference.

---

ChestXRay v3.2 - NIH ChestX-ray14 Multi-Label Pathology Detection

Overview

Research-grade implementation of multi-label chest X-ray pathology detection using EfficientNet-B4 on the full NIH ChestX-ray14 dataset (112,120 images) with official patient-level splits.

Version: 3.2
Developed by: FratresMedAI
Status: Critical fixes implemented, conservative performance expectations documented
Compatible with: RunPod PyTorch 2.8.0 (CUDA 12.8.1)

---

Quick Start

1. Launch RunPod Pod

# Recommended template
runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404

# GPU requirements
- H200 (141GB VRAM) - recommended for batch_size=48
- H100 (80GB VRAM) - use batch_size=32

# Storage requirements
- Container disk: 200GB minimum (for full dataset)

2. Upload Notebook

Upload ChestXRay_PD_GPU_v3.2_FIXED.ipynb to RunPod Jupyter environment.

3. Cell Execution Order

Run cells sequentially:

Cell	Description	Expected Time	Notes
0	Bootstrap	3-5 minutes	Installs dependencies, no torch reinstall
1	Environment + Config	<1 minute	Initialize wandb here
2	Dataset Download	60-90 minutes	All 12 zips with retry logic
3	Data Exploration	2-3 minutes	Official splits verification
4	Dataset + Dataloaders	1-2 minutes	CutMix before transforms
5	Model + Loss	1-2 minutes	EfficientNet-B4 or timm backbones
6	Training Loop	180-240 minutes	50 epochs, early stopping
7	Temperature Scaling	5-10 minutes	Calibration on validation set
8	Test Evaluation	15-20 minutes	5-fold TTA + bootstrapped CIs
9	Visualization	2-3 minutes	6-panel metrics plot
10	Final Summary	<1 minute	Artifact checklist

Total runtime: 4-5 hours end-to-end

---

Performance Expectations

Conservative Targets (Literature-Backed)

Configuration	Expected Mean AUC	Notes	Reference
EfficientNet-B4 (default)	0.805-0.820	Baseline, well-executed	[1,2]
EfficientNet-B4 + Asymmetric Loss	0.810-0.825	Better for rare classes	[1,3]
ConvNeXt-Base + Asymmetric Loss	0.830-0.845	Publication-competitive	[4,5]
EfficientNetV2-M + Asymmetric Loss	0.825-0.840	Faster training	[2,6]

Interpretation Guidelines

• 0.805-0.820: Expected for EfficientNet-B4 (matches literature [1,2])
• 0.820-0.830: Strong result, above typical EfficientNet-B4
• 0.830-0.840: Very strong, competitive with best single models [4,5]
• 0.840+: Publication-worthy (requires ConvNeXt/CoAtNet or novel methods)

Performance Claims - Concrete References

[1] Rajpurkar et al. (2017) "CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning" - Established baseline DenseNet-121 achieving 0.8094 mean AUC on ChestX-ray14

[2] Ucan (2025) "Comparison of EfficientNet CNN models for multi-label chest X-ray disease diagnosis" PeerJ Comput Sci - Mean AUC ~0.807-0.8265 for EfficientNet variants

[3] Ridnik et al. (2021) "Asymmetric Loss For Multi-Label Classification" (ICCV) - Demonstrates 2-3% AUC improvement on medical imaging tasks with extreme imbalance

[4] Liu et al. (2022) "A ConvNet for the 2020s" (CVPR) - ConvNeXt architecture achieving state-of-art on multiple vision benchmarks

[5] Xiong et al. (2025) "Multi-Label Disease Detection in Chest X-Ray Imaging Using a Fine-Tuned ConvNeXtV2" MDPI Informatics - Mean AUC 0.8523 on ChestXray14

[6] Tan & Le (2021) "EfficientNetV2: Smaller Models and Faster Training" (ICML) - 20-30% faster training than EfficientNet-B4 with comparable accuracy

Note: Results may vary ±0.005 AUC due to GPU non-determinism even with fixed seeds. For exact reproduction, document GPU model, driver version, and CUDA version.

---

Configuration Options

Switch to ConvNeXt for Publication (Target: 0.83+)

In CELL 1, modify:

# For publication-competitive results
BACKBONE = 'convnext_base'  # Instead of 'efficientnet_b4'
USE_TIMM = True  # Enable timm library
LOSS_TYPE = 'hybrid_bce_asymmetric'  # Better for extreme imbalance
BATCH_SIZE = 24  # ConvNeXt needs more VRAM, reduce if OOM

Expected performance with ConvNeXt:

• Mean AUC: 0.830-0.845
• Hernia AUC: 0.75-0.84
• Training time: 240-300 minutes (larger model)

Switch to EfficientNetV2 (Faster Training)

BACKBONE = 'efficientnetv2_m'
USE_TIMM = True
LOSS_TYPE = 'hybrid_bce_asymmetric'
BATCH_SIZE = 32

Expected performance:

• Mean AUC: 0.825-0.840
• Training time: 150-200 minutes (faster than B4)

Enable/Disable AMP (Automatic Mixed Precision)

In CELL 1:

USE_AMP = True  # Default: enabled for ~20-30% speedup

AMP is already integrated in training loop (CELL 6) via torch.cuda.amp.autocast().

Adjust Batch Size for VRAM

# H200 (141GB)
BATCH_SIZE = 48  # Safe for 336x336
GRAD_ACCUM_STEPS = 4  # Effective batch 192

# H100 (80GB)
BATCH_SIZE = 32  # Conservative for 336x336
GRAD_ACCUM_STEPS = 6  # Effective batch 192

# A100 (40GB) or OOM issues
BATCH_SIZE = 16
GRAD_ACCUM_STEPS = 12  # Effective batch 192

---

Wandb Integration

Setup

1. Get API key: https://wandb.ai/authorize
2. In CELL 1, wandb.init() will prompt for key on first run
3. Project name: chestxray-efficientnet-v3.2

Sharing Results

After training completes:

# Get wandb run URL
print(wandb.run.get_url())

Share this URL for:

• Live training metrics
• Hyperparameter configuration
• Artifact downloads (models, predictions, plots)

Disable Wandb (Optional)

# In CELL 1, comment out:
# wandb.init(...)

---

Dataset Download Fallback

If NIH Box.com throttles (common after 2-3 files):

Option 1: Kaggle CLI (Faster)

# In CELL 2, before download section
!pip install kaggle
!mkdir -p ~/.kaggle
# Upload kaggle.json to ~/.kaggle/
!kaggle datasets download -d nih-chest-xrays/data
!unzip data.zip -d /workspace/data/images

Option 2: Manual Download

1. Download from: https://nihcc.app.box.com/v/ChestXray-NIHCC
2. Upload all 12 zips to /workspace/data/
3. Run extraction section of CELL 2

---

VRAM Monitoring

After CELL 5 (model creation), check VRAM usage:

# Quick VRAM test
import torch
dummy_batch = torch.randn(BATCH_SIZE, 3, INPUT_SIZE, INPUT_SIZE).to(DEVICE)
with torch.no_grad():
    _ = model(dummy_batch)
peak_vram = torch.cuda.max_memory_allocated() / 1e9
print(f"Peak VRAM: {peak_vram:.1f} GB")

# Expected ranges:
# - Batch 32, 336x336: 30-35 GB (safe for H100)
# - Batch 48, 336x336: 40-45 GB (needs H200)
# - If >60 GB: reduce batch size

---

How to Reproduce Exact Results

Reproducibility Checklist

1. Fixed Random Seed

# Already set in CELL 1
SEED = 42
torch.manual_seed(SEED)
torch.cuda.manual_seed_all(SEED)
np.random.seed(SEED)
random.seed(SEED)

2. Deterministic Mode

# Already enabled in CELL 1
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = True
torch.use_deterministic_algorithms(True)  # May reduce performance slightly

3. Document Exact Environment

• GPU: H200 SXM (141GB) or H100 (80GB)
• CUDA: 12.8.1
• PyTorch: 2.8.0
• Driver: Document with nvidia-smi
• Container: runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404

4. Expected Variability

• Results may vary ±0.005 AUC across runs due to:
  • GPU non-determinism in some CUDA operations
  • Floating-point precision differences
  • CuDNN algorithm selection

5. Bit-Exact Reproduction For bit-exact reproduction (not recommended due to speed):

# Disable cudnn benchmark
torch.backends.cudnn.benchmark = False
# Use CPU (very slow)
DEVICE = torch.device('cpu')

6. Save Run Metadata After training, document:

• Exact GPU model and driver version
• Final mean AUC and per-class AUCs
• Training time and number of epochs
• Wandb run URL for full hyperparameters

---

Example Run Results

Placeholder - Fill After First Training Run

Configuration: EfficientNet-B4, Batch 32, Official Splits, Full Dataset

Metric	Value	95% CI Lower	95% CI Upper	Notes
Mean AUC	TBD	TBD	TBD	Target: 0.805-0.820
Atelectasis AUC	TBD	TBD	TBD
Cardiomegaly AUC	TBD	TBD	TBD
Effusion AUC	TBD	TBD	TBD
Infiltration AUC	TBD	TBD	TBD
Mass AUC	TBD	TBD	TBD
Nodule AUC	TBD	TBD	TBD
Pneumonia AUC	TBD	TBD	TBD
Pneumothorax AUC	TBD	TBD	TBD
Consolidation AUC	TBD	TBD	TBD
Edema AUC	TBD	TBD	TBD
Emphysema AUC	TBD	TBD	TBD
Fibrosis AUC	TBD	TBD	TBD
Pleural_Thickening AUC	TBD	TBD	TBD
Hernia AUC	TBD	TBD	TBD	Target: 0.72-0.81
Mean F1	TBD	TBD	TBD
Mean ECE	TBD	-	-	Target: <0.05
Training Time	TBD	-	-	Expected: 180-240 min
Final Epoch	TBD	-	-	Max: 50

Environment:

• GPU: [Document after run]
• CUDA: [Document after run]
• Driver: [Document after run]
• Date: [Document after run]
• Wandb Run: [Insert URL after run]

Instructions: After completing your first training run, fill this table with actual results from test_metrics_v3.2.csv and update README for future reference.

---

Output Artifacts

All artifacts saved to /workspace/results/:

1. test_metrics_v3.2.csv - Per-class AUC, F1, precision, recall with 95% CIs
2. optimal_thresholds_v3.2.csv - Youden's J optimal thresholds per class
3. test_predictions_tta_v3.2.csv - TTA-averaged predictions + uncertainties
4. comprehensive_metrics_v3.2.png - 6-panel visualization
5. Model checkpoint: /workspace/models/best_model_v3.2.pth

---

Troubleshooting

OOM (Out of Memory) Error

# In CELL 1, reduce batch size
BATCH_SIZE = 16  # or 24
GRAD_ACCUM_STEPS = 12  # Keep effective batch ~192

Box.com Download Timeout

Use Kaggle CLI fallback (see Dataset Download Fallback section above).

Wandb Login Issues

# Manual login
import wandb
wandb.login(key='YOUR_API_KEY')

Training Too Slow

# Enable cudnn benchmark (already in CELL 1)
torch.backends.cudnn.benchmark = True

# Or switch to EfficientNetV2 (faster)
BACKBONE = 'efficientnetv2_m'
USE_TIMM = True

---

Disclaimer

THIS IS A RESEARCH TOOL, NOT A DIAGNOSTIC DEVICE.

• NOT FOR CLINICAL USE
• NOT FDA-APPROVED
• FOR RESEARCH AND EDUCATIONAL PURPOSES ONLY
• NOT INTENDED FOR MEDICAL DIAGNOSIS OR TREATMENT DECISIONS

This implementation is provided for academic research and portfolio demonstration. Any clinical application requires extensive validation, regulatory approval, and clinical trials.

---

Citation

If using this notebook for research:

@misc{chestxray_v32_2026,
  title={Multi-Label Chest X-Ray Pathology Detection with EfficientNet-B4},
  author={FratresMedAI},
  year={2026},
  note={Implementation based on NIH ChestX-ray14 dataset with official patient-level splits}
}

Dataset citation:

@article{wang2017chestxray,
  title={ChestX-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases},
  author={Wang, Xiaosong and Peng, Yifan and Lu, Le and Lu, Zhiyong and Bagheri, Mohammadhadi and Summers, Ronald M},
  journal={CVPR},
  year={2017}
}

---

References

Official Dataset

• NIH ChestX-ray14: https://nihcc.app.box.com/v/ChestXray-NIHCC
• Official splits: https://github.com/zoogzog/chexnet

Key Papers (2025-2026)

• EfficientNet-B4/B5 on ChestX-ray14: 0.805-0.818 mean AUC
• ConvNeXtV2 for medical imaging: 0.840-0.852 mean AUC
• Asymmetric Loss for extreme imbalance: CVPR 2025

RunPod Image

• runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404
• PyTorch 2.8.0 + CUDA 12.8.1 + Ubuntu 24.04

---

Last Updated: March 16, 2026
Version: 3.2
Developed by: FratresMedAI
Maintainer: FratresMedAI