Lapvqa Ad

--- tags: - chest-xray - radiology - object-detection - abnormality-detection - vindr-cxr license: apache-2.0 --- # LAPVQA — Abnormality Detection Part of the [LAPVQA collection](https://huggingface.co/collections/dmusingu/lapvqa). ## Description DETR-style detection heads for 14-class chest abnormality detection on VinDr-CXR, trained on top of six **frozen** vision encoders. Each checkpoint is a dict: `{state_dict, vis_dim, d_model, num_queries, num_enc, num_dec, encoder, epoch, val_map40, val_map50}`. ## Architecture — `DetectionHead` ``` vis_proj : Linear(vis_dim → 256) encoder : 2 × TransformerEncoderLayer (self-attn, pre-norm) object_queries : Parameter [1, 20, 256] decoder : 3 × TransformerDecoderLayer (cross-attn to encoder output) class_head : Linear(256 → 15) # 14 classes + background box_head : MLP(256 → 256 → 4) # (cx,cy,w,h) ∈ [0,1] ``` ## Results (VinDr-CXR test, mAP@IoU=0.4) | Encoder | [email protected] (test) | |---|---| | OWLv2 | 0.048 | | SigLIP | ~0.045 | | CLIP ViT-L/14 | ~0.040 | | File | Encoder | vis_dim | |---|---|---| | `clip-vit-l14.pt` | CLIP ViT-L/14 | 1024 | | `siglip.pt` | SigLIP | 1152 | | `florence2.pt` | Florence-2 | 1024 | | `coca.pt` | CoCa | 768 | | `owlv2.pt` | OWLv2 | 1024 | | `mae-vit-l16.pt` | MAE ViT-L/16 | 1024 | ## Loading ```python import torch from lapvqa.ad.heads import DetectionHead from lapvqa.ad.heads import predict ckpt = torch.load("owlv2.pt", map_location="cpu") head = DetectionHead( vis_dim = ckpt["vis_dim"], d_model = ckpt["d_model"], num_queries = ckpt["num_queries"], num_enc_layers = ckpt["num_enc"], num_dec_layers = ckpt["num_dec"], ) head.load_state_dict(ckpt["state_dict"]) head.eval() with torch.no_grad(): # vis_tokens: [B, HW, vis_dim] — spatial patch tokens from the frozen encoder outputs = head(vis_tokens) detections = predict(outputs, score_threshold=0.1, nms_iou=0.5) # detections[i]: {'boxes': [K,4] xyxy, 'labels': [K], 'scores': [K]} ```

Source context: 0 downloads · 0 likes · Pipeline object-detection · Repo dmusingu/lapvqa-ad