Scribble Segmentation

--- license: mit library_name: pytorch tags: - image-segmentation - scribble-supervised - pascal-voc - u-net - ensemble pipeline_tag: image-segmentation datasets: - pascal-voc metrics: - miou model-index: - name: scribble-segmentation results: - task: type: image-segmentation name: Scribble-supervised binary segmentation dataset: type: pascal-voc name: PASCAL VOC scribble subset (228 train, 226 test1, 228 test2) metrics: - type: miou value: 0.842 name: Mean IoU (5-fold out-of-fold) - type: bg_iou value: 0.925 name: Background IoU - type: fg_iou value: 0.760 name: Foreground IoU --- # Scribble Segmentation Ensemble Binary foreground/background segmentation from sparse user scribbles. Honest cross-validated mean IoU of 0.842 on the PASCAL VOC scribble subset, trained from scratch with no pretrained weights. The pipeline combines two five-fold ensembles of small U-Nets averaged with multi-scale test-time augmentation, then calibrated by a per-image threshold model. ## Results | Method | Mean IoU | Background IoU | Foreground IoU | |---|---|---|---| | Per-image K-NN baseline (k=11) | 0.499 | 0.637 | 0.361 | | First U-Net (no CutMix) | 0.788 | 0.900 | 0.676 | | U-Net trained with CutMix augmentation | 0.819 | 0.913 | 0.724 | | Pair of CutMix U-Nets (different random seeds), averaged | 0.832 | 0.919 | 0.743 | | **CutMix U-Net averaged with a U-Net trained on pseudo-labels (this release)** | **0.842** | **0.925** | **0.760** | The progression reads top to bottom. Switching from per-image K-NN to a single globally-trained U-Net is the largest jump because the U-Net learns from every pixel of the 228 ground-truth masks instead of just the sparse scribbles. CutMix augmentation gives the next bump by spatially recombining training examples, which matters on a small dataset like this one. Averaging two seed twins removes some of the variance between any single model's mistakes. The final step replaces one seed twin with a U-Net that also saw 454 unlabeled test images with predicted labels from the previous ensemble. The pseudo-labels are noisy (around 17% wrong on average) but the extra visual diversity wins by about 0.01 mIoU. For context: the original course leaderboard had 28 teams. This release would place top four. The winning team reached 0.868. ## Quick start The two essential inference scripts (`predict_ensemble.py` and `train...

Source context: 0 downloads · 1 likes · Pipeline image-segmentation · Library pytorch · Repo Enorenio/scribble-segmentation