Medical Models

Medical Imaging Models

This repository contains two complementary medical imaging AI model collections:

1. BiomedCLIP Router - Routes medical images to the correct imaging modality
2. Disease-Specific Models - Classification and segmentation models for various medical imaging domains

---

Model 1: BiomedCLIP Router

Description

A modality classification model that routes medical images to one of 7 imaging categories. Built on Microsoft's BiomedCLIP backbone, this router enables automatic sorting of medical images to the appropriate downstream model.

Supported Modalities

Class	Modality
0	Endoscopy
1	Dermatology
2	X-Ray
3	Ultrasound
4	Mammography
5	Fundus / Retinography
6	Microscopy

Performance

Metric	Test Score
Accuracy	100%
Macro F1	100%
Macro Precision	100%
Macro Recall	100%

Training Details

• Base Model: microsoft/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224
• Training Images: 14,000 (2,000 per class)
• Validation Images: 1,750 (250 per class)
• Test Images: 1,750 (250 per class)
• Best Epoch: 1
• Validation Macro F1: 99.94%

Files

biomedclip_router_20260423_194004/
├── best_biomedclip_router.pt    # Trained model weights (748MB)
├── metrics.json                  # Training metrics and history
├── train.log                     # Training logs
├── session_name.txt              # Session identifier
└── unseen_holdout_300/          # Holdout test data info

---

Model 2: Disease-Specific Models

A collection of specialized models for disease classification and segmentation across 8 medical imaging domains.

---

2.1 Breast Ultrasound

Classification Model

Task: Classify breast ultrasound images as benign, malignant, or normal

Metric	Test Score
Accuracy	92.31%
Macro F1	90.75%
Macro Precision	89.79%
Macro Recall	91.85%
Best Epoch	23

Classes: benign, malignant, normal

Segmentation Model

Task: Segment lesions in breast ultrasound images

Metric	Test Score
Dice	83.80%
IoU	76.17%
Best Epoch	20
Architecture	SMP UNet++

Files:

breast_ultrasound/
├── classification/
│   ├── best_classifier.pt
│   ├── metrics.json
│   └── plots/
└── segmentation/
    ├── best_segmentor.pt
    ├── metrics.json
    └── plots/

---

2.2 Chest X-Ray

Classification Model

Task: 4-class chest X-ray classification (COVID-19 Radiography Database)

Metric	Test Score
Accuracy	96.17%
Macro F1	96.08%
Macro Precision	96.40%
Macro Recall	95.79%
Best Epoch	11

Classes: covid, lung_opacity, normal, viral_pneumonia

• Architecture: DenseNet121 (torchvision IMAGENET1K_V1, the CheXNet chest-X-ray backbone) + fresh 4-class head (replaces the previous 15-class BiomedCLIP multi-label model that scored ~21% exact-match accuracy)
• Dataset: COVID-19 Radiography Database (Kaggle tawsifurrahman/covid19-radiography-database); also mirrored at dataset repo umairinayat/fyp-dataset
• Deduplication: near-duplicates removed via MD5 + perceptual pHash (mirror-aware) -> 19,133 unique images
• Split: stratified 70/15/15 (train 13,393 / val 2,870 / test 2,870), deduplicated globally before splitting (no train/test leakage)
• Input: 224x224, ImageNet normalization
• Training: 15 epochs, AdamW + OneCycleLR (max_lr 2e-4, weight_decay 0.03), tempered class-weighted CrossEntropy (label smoothing 0.1), mixed precision; augmentation = RandomResizedCrop + TrivialAugmentWide + flip + RandomErasing (train only); no test-time augmentation

Images per class (after dedup; total unique = 19,133):

Class	Train	Val	Test	Total
covid	2,225	477	477	3,179
lung_opacity	4,088	876	876	5,840
normal	6,152	1,318	1,319	8,789
viral_pneumonia	928	199	198	1,325
Total	13,393	2,870	2,870	19,133

Segmentation Model

Task: Binary lung-region segmentation (using the dataset's masks/)

Metric	Test Score
Dice	98.71%
IoU	97.50%
Best Epoch	15
Architecture	SMP UNet (EfficientNet-B0 ImageNet encoder)

• Loss: BCEWithLogits + Dice; same leakage-free split as classification; image & mask resized to 256x256; train augmentation = flip + rot90

Files:

chest_xray/
├── classification/
│   ├── best_classifier.pt
│   ├── metrics.json
│   ├── training_history.json
│   └── plots/
└── segmentation/
    ├── best_segmenter.pt
    ├── metrics.json
    ├── training_history.json
    └── plots/

---

2.3 Dermatology

Classification Model

Task: Classify skin lesions as malignant or non-malignant (2-class)

Metric	Test Score
Accuracy	92.25%
Macro F1	90.47%
Macro Precision	89.50%
Macro Recall	91.63%
Best Epoch	8

Classes: malignant, non_malignant

• Architecture: ConvNeXt-Small (torchvision IMAGENET1K_V1) + 2-class head (replaces the previous BiomedCLIP model at 88.74%)
• Dataset: combined 2 sources for variety - HAM10000 (Kaggle surajghuwalewala/ham1000-segmentation-and-classification, 7->2 class) + fanconic/skin-cancer-malignant-vs-benign
• Leakage-free split: HAM10000 has multiple images per lesion, so it is split by lesion_id (all images of a lesion stay in one split); fanconic uses its own official train/test holdout -> train 9,667 / val 1,489 / test 2,156
• Training: 12 epochs @ 224px, AdamW + OneCycleLR, tempered class-weighted CE (label smoothing 0.1), mixed precision; augmentation = RandomResizedCrop + TrivialAugmentWide + flip + RandomErasing

Images per class:

Class	Train	Val	Test	Total
malignant	2,559	307	585	3,451
non_malignant	7,108	1,182	1,571	9,861
Total	9,667	1,489	2,156	13,312

Segmentation Model

Task: Binary lesion segmentation (HAM10000 lesion masks)

Metric	Test Score
Dice	95.25%
IoU	91.52%
Best Epoch	18
Architecture	SegFormer-B3 (transformer)

• Architecture: SegFormer-B3 (nvidia/segformer-b3-finetuned-ade-512-512, ADE20K-pretrained) with a 1-label binary head
• Dataset: HAM10000 lesion masks (Kaggle surajghuwalewala/ham1000-segmentation-and-classification), 10,015 image+mask pairs
• Split: lesion-level (by lesion_id) so the same lesion never appears in both train and test -> train 6,987 / val 1,478 / test 1,550 (no leakage)
• Training: 18 epochs @ 256px, AdamW + OneCycleLR (max_lr 6e-5), BCEWithLogits + Dice, mixed precision; augmentation = hflip + rot90

Files:

dermatology/
├── classification/
│   ├── best_classifier.pt
│   ├── metrics.json
│   └── plots/
└── segmentation/
    ├── best_segmenter.pt
    ├── metrics.json
    └── plots/

---

2.4 Endoscopy

Classification Model

Task: Classify GI endoscopy images into 5 classes (healthy added vs. previous 4-class model)

Metric	Test Score
Accuracy	94.73%
Macro F1	80.35%
Macro Precision	81.96%
Macro Recall	79.66%
Best Epoch	20

Classes: barretts, esophagitis, polyp, ulcerative_colitis, healthy

• Architecture: ConvNeXt-Small (torchvision IMAGENET1K_V1) + fresh 5-class head (replaces the previous 4-class BiomedCLIP model)
• Dataset: HyperKvasir labeled images (23 source classes mapped to 5 targets); barretts<-{barretts,barretts-short-segment}, esophagitis<-{esophagitis-a,esophagitis-b-d}, polyp<-{polyps}, ulcerative_colitis<-{ulcerative-colitis-grade-*}, healthy<-{cecum,pylorus,z-line,retroflex-rectum,retroflex-stomach,ileum}
• Deduplication: near-duplicates removed via MD5 + perceptual pHash (mirror-aware) -> 6,695 unique images
• Split: stratified 70/15/15 (train 4,686 / val 1,004 / test 1,005), deduplicated globally before splitting (no train/test leakage)
• Input: 384x384, ImageNet normalization
• Training: 22 epochs, AdamW + OneCycleLR (max_lr 2e-4, weight_decay 0.05), tempered class-weighted CrossEntropy (label smoothing 0.1), mixed precision; augmentation = RandomResizedCrop + TrivialAugmentWide + flip + RandomErasing (train only)
• Eval: no test-time augmentation (single center-crop)
• Note: barretts has only 94 images in all of HyperKvasir (66 train / 14 test), so it remains the hardest class and caps macro-F1; the other 4 classes score F1 0.84-0.99

Images per class (after dedup; total unique = 6,695):

Class	Train	Val	Test	Total
barretts	66	14	14	94
esophagitis	463	100	99	662
polyp	713	152	153	1,018
ulcerative_colitis	587	126	126	839
healthy	2,857	612	613	4,082
Total	4,686	1,004	1,005	6,695

Segmentation Model

Task: Segment regions of interest in endoscopy images

Metric	Test Score
Dice	88.52%
IoU	82.18%
Best Epoch	38
Architecture	SMP UNet++

Files:

endoscopy/
├── classification/
│   ├── best_classifier.pt
│   ├── metrics.json
│   └── plots/
└── segmentation/
    ├── best_segmentor.pt
    ├── metrics.json
    └── plots/

---

2.5 Mammography

Classification Model

Task: Classify mammogram patches as benign, malignant, or normal (3-class)

Metric	Test Score
Accuracy	99.63%
Macro F1	99.72%
Macro Precision	99.71%
Macro Recall	99.72%
Best Epoch	12

Classes: benign, malignant, normal

• Architecture: ResNet50 (torchvision IMAGENET1K_V2 pretrained weights) + fresh 3-class linear head
• Dataset: INbreast + MIAS + DDSM (CLAHE preprocessed) - 26,602 raw images
• Deduplication: 10,472 duplicate / near-duplicate images removed via MD5 + perceptual pHash (LSH banding, mirror-aware, Hamming <= 5) -> 16,130 unique images
• Split: stratified 70/15/15 (train 11,291 / val 2,419 / test 2,420), deduplicated globally before splitting, so the held-out test set contains no image (or near-duplicate) seen in train/val - no leakage
• Input: 224x224, grayscale replicated to 3 channels, ImageNet normalization
• Training: 15 epochs, AdamW + OneCycleLR (max_lr 3e-4), class-weighted CrossEntropy, mixed precision
• Augmentation (train only): RandomResizedCrop, RandomHorizontalFlip, RandomRotation(15), RandomAffine, ColorJitter, RandomErasing

Images per class (after dedup; total unique = 16,130):

Class	Train	Val	Test	Total
benign	4,505	965	966	6,436
malignant	5,428	1,163	1,163	7,754
normal	1,358	291	291	1,940
Total	11,291	2,419	2,420	16,130

Replaces the previous 2-class (BENIGN/MALIGNANT) BiomedCLIP mammography classifier (was 66% accuracy). The new model is a ResNet50 - load it with torchvision.models.resnet50 (see Usage below), not the BiomedCLIP loader.

Test confusion matrix (rows = true, cols = predicted benign, malignant, normal):

benign:    [963,    3,    0]
malignant: [  6, 1157,    0]
normal:    [  0,    0,  291]

Segmentation Model

Task: Binary mass segmentation in mammography (CBIS-DDSM)

|...