Image Context Sorter Dinov2

--- license: other license_name: dinov2-derivative tags: - image-classification - dinov2 - timm - pytorch library_name: timm pipeline_tag: image-classification --- # Image Context Sorter — DINOv2 partial fine-tune Restoration-work photo classifier for 하우스캐치 (water-damage restoration). Classifies a single photo into one of 8 work-type / stage categories. **Private / internal** — trained on client work photos; not for redistribution. ## Results (locked test set, 8 classes) | Model | macro-F1 | top-2 | |-------|---------:|------:| | 518px frozen linear probe (baseline) | 0.601 | 0.830 | | 224px frozen linear probe (control) | 0.617 | 0.800 | | **224px partial fine-tune (this model)** | **0.737** | **0.895** | Every class improved over the probe; the previously weakest class (곰팡이제거) rose 0.44 → 0.67 and its 곰팡이제거↔누수탐지 confusion was eliminated. ## Model - **Base:** `vit_base_patch14_dinov2.lvd142m` (timm / Meta DINOv2) - **Fine-tune:** partial — the last **4** transformer blocks + final norm + a new 8-class head are unfrozen; the rest of the backbone is frozen. Input **224px** (position-embedding interpolation). Discriminative LR (backbone 1e-5 / head 1e-3), AdamW + cosine schedule, float16 autocast (Apple MPS), class-weighted cross-entropy, early stopping on val macro-F1. **No vertical-flip** augmentation (ceiling/floor structure is a label cue). - **Calibration:** temperature scaling, T ≈ 2.44 (fit on the val split). Apply it to logits before softmax for honest confidence; it does not change argmax/accuracy. ## Classes (index order) `0 철거중 · 1 비포1 · 2 애프터 · 3 보양작업 · 4 석고보드설치 · 5 누수탐지 · 6 도배작업 · 7 곰팡이제거` ## How to load ```python import timm, torch from torchvision import transforms as T model = timm.create_model( "vit_base_patch14_dinov2.lvd142m", pretrained=False, num_classes=8, img_size=224 ) model.load_state_dict(torch.load("finetune.pt", map_location="cpu")) model.eval() tf = T.Compose([ T.Resize(224), T.CenterCrop(224), T.ToTensor(), T.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)), ]) # logits = model(tf(img).unsqueeze(0)) # probs = torch.softmax(logits / 2.44, dim=1) # temperature-calibrated ``` ## License This is a **derivative of Meta's DINOv2** (`vit_base_patch14_dinov2.lvd142m`). Use is governed by the DINOv2 model license — see <https://github.com/facebookresearch/dinov2>. Confirm redistribution / commercial t...