Drywall Qa Clipseg

--- license: mit language: - en tags: - clipseg - segmentation - construction - drywall - quality-assurance - text-conditioned - binary-mask library_name: transformers base_model: CIDAS/clipseg-rd64-refined pipeline_tag: image-segmentation datasets: - roboflow/drywall-join-detect - roboflow/cracks-3ii36 metrics: - iou - dice --- <p align="center"> <h1 align="center">Prompted Segmentation for Drywall QA</h1> <p align="center"> Text-conditioned binary mask prediction for construction defect detection </p> </p>        <p align="center"> <a href="#1-methodology">Methodology</a> &bull; <a href="#2-data-preparation">Data Preparation</a> &bull; <a href="#3-results">Results</a> &bull; <a href="#4-failure-cases--potential-solutions">Failure Cases</a> &bull; <a href="#quick-start">Quick Start</a> &bull; Full Report (PDF) </p> --- Feed a construction photo and a text prompt. Get a binary segmentation mask back. Two tasks — **crack detection** and **drywall taping/joint detection** — both driven by natural language at inference time. Change the prompt, change what gets segmented. No class heads, no retraining. ``` Input: image.jpg + "segment wall crack" Output: image__segment_wall_crack.png (binary mask, {0, 255}) ``` --- ## 1. Methodology ### Model: CLIPSeg We fine-tune [**CLIPSeg**](https://arxiv.org/abs/2112.10003) (Luddecke & Ecker, CVPR 2022) — a text-conditioned segmentation model built on CLIP. The entire CLIP backbone (149.6M params) stays **frozen**. Only a lightweight 3-block transformer decoder with U-Net skip connections (**1.13M params**) is trained.  The model takes an RGB image and a text prompt. The CLIP vision encoder (ViT-B/16) and text encoder indepen...