Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
LoRA-adapted version of CIDAS/clipseg-rd64-refined for cloud segmentation on Sentinel-2 satellite imagery using the CloudSEN12+ dataset. This model is part of the research presented in:
Model source
Source description
LoRA-adapted version of CIDAS/clipseg-rd64-refined for cloud segmentation on Sentinel-2 satellite imagery using the CloudSEN12+ dataset. This model is part of the research presented in:
Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift Harshith Kethavath, Weiming Hu EarthVision Workshop @ CVPR 2026
All models from this paper:
Sources
1 sourceVerified Aug 18
Model artifacts
1 artifactSource excerpts
2 excerptsCLIPSeg is a vision-language segmentation model trained on natural images. This variant uses Low-Rank Adaptation (LoRA) to adapt CLIPSeg to Sentinel-2 satellite imagery for four-class cloud segmentation: clear, thick cloud, thin cloud, and cloud shadow. Compared to full fine-tuning, LoRA trains only a small fraction of parameters (~16MB adapter weights vs. ~603MB full model), making it a lightweight alternative.
from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation
from peft import PeftModel
import torch
from PIL import Image
base_model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined")
model = PeftModel.from_pretrained(base_model, "uga-gaim/CLIPSeg-CloudSEN12Plus-LoRA")
processor = CLIPSegProcessor.from_pretrained("uga-gaim/CLIPSeg-CloudSEN12Plus-LoRA")
image = Image.open("your_sentinel2_image.png")
prompts = ["clear", "thick cloud", "thin cloud", "cloud shadow"]
inputs = processor(
text=prompts,
images=[image] * len(prompts),
return_tensors="pt",
padding=True
)
with torch.no_grad():
outputs = model(**inputs)
logits = outputs.logits # shape: (4, H, W)
predicted_class = logits.argmax(dim=0) # per-pixel class prediction
Trained on the CloudSEN12+ dataset, the largest expert-labeled cloud segmentation benchmark for Sentinel-2 imagery. 100% of the training split was used (full data setting).
| Hyperparameter | Value |
|---|---|
| Optimizer | AdamW |
| Learning rate | 2e-4 |
| Weight decay | 0.01 |
| Warmup ratio | 0.03 |
| Epochs | 15 |
| Batch size | 16 |
| LoRA rank (r) | 32 |
| LoRA alpha (α) | 64 |
| LoRA dropout | 0.05 |
| Precision | fp16 |
Combined segmentation loss: weighted sum of Focal loss, Tversky loss, and Boundary loss.
Evaluated on the CloudSEN12+ test split. Per-class IoU:
| Class | Zero-Shot (baseline) | This model (LoRA 100%) |
|---|---|---|
| Clear | 0.5205 | 0.8269 |
| Thick Cloud | 0.2773 | 0.7488 |
| Thin Cloud | 0.0898 | 0.3820 |
| Cloud Shadow | 0.1325 | 0.4389 |
| mIoU | 0.2550 | 0.5991 |
@InProceedings{Kethavath_2026_CVPR,
author = {Kethavath, Harshith and Hu, Weiming},
title = {Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2026},
pages = {7960-7969}
}
--- license: cc-by-4.0 datasets: - isp-uv-es/CloudSEN12Plus base_model: - CIDAS/clipseg-rd64-refined language: - en pipeline_tag: image-segmentation tags: - cloud - segmentation - remote-sensing - satellite - earth-observation - lora - peft --- # CLIPSeg Fine-tuned for Cloud Segmentation (LoRA, 100% Data) LoRA-adapted version of [CIDAS/clipseg-rd64-refined](https://huggingface.co/CIDAS/clipseg-rd64-refined) for cloud segmentation on Sentinel-2 satellite imagery using the CloudSEN12+ dataset. This model is part of the research presented in: > **Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift** > Harshith Kethavath, Weiming Hu > EarthVision Workshop @ CVPR 2026 All models from this paper: [https://huggingface.co/collections/uga-gaim/2026-cloudprompts](https://huggingface.co/collections/uga-gaim/2026-cloudprompts) ## Model Description CLIPSeg is a vision-language segmentation model trained on natural images. This variant uses Low-Rank Adaptation (LoRA) to adapt CLIPSeg to Sentinel-2 satellite imagery for four-class cloud segmentation: clear, thick cloud, thin cloud, and cloud shadow. Compared to full fine-tuning, LoRA trains only a small fraction of parameters (~16MB adapter weights vs. ~603MB full model), making it a lightweight alternative. - **Developed by:** Harshith Kethavath, Weiming Hu - **Lab:** Lab for Geoinformatics and AI Modeling (GAIM), University of Georgia - **License:** CC BY 4.0 - **Base model:** CIDAS/clipseg-rd64-refined - **PEFT method:** LoRA (rank 32, α = 64) ## How to Get Started ```python from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation from peft import PeftModel import torch from PIL import Image base_model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined") model = PeftModel.from_pretrained(base_model, "uga-gaim/CLIPSeg-CloudSEN12Plus-LoRA") processor = CLIPSegProcessor.from_pretrained("uga-gaim/CLIPSeg-CloudSEN12Plus-LoRA") image = Image.open("your_sentinel2_image.png") prompts = ["clear", "thick cloud", "thin cloud", "cloud shadow"] inputs = processor( text=prompts, images=[image] * len(prompts), return_tensors="pt", padding=True ) with torch.no_grad(): outputs = model(**inputs) logits = outputs.logits # shape: (4, H, W) predicted_class = logits.argmax(dim=0) # per-pixel class prediction ``` ## Training Details ### Tra...
Source context: 25 downloads · 1 likes · Pipeline image-segmentation · Library peft · Repo uga-gaim/CLIPSeg-CloudSEN12Plus-LoRA