ModelPrésentéExterne

CLIPSeg CloudSEN12Plus FFT

Fine-tuned version of CIDAS/clipseg-rd64-refined for cloud segmentation on Sentinel-2 satellite imagery using the CloudSEN12+ dataset. This model is part of the research presented in:

Sourcehuggingface.co/uga-gaim/CLIPSeg-CloudSEN12Plus-FFT/tree/34ea01f8b42d0fdee389fb8e3b5fa5b7f53158e3

TypeText modelModèle de baseCIDAS/clipseg-rd64-refinedVersionFP32BClasse

Source du modèle

Type: Text model
Taille du fichier: 575 MB
Modèle de base: CIDAS/clipseg-rd64-refined
Version: vFP32
Mis à jour: 1 nov. 2026
Téléchargements: 46
Source: Hugging Face

Description de la source

CLIPSeg Fine-tuned for Cloud Segmentation (Full Fine-Tuning, 100% Data)

Fine-tuned version of CIDAS/clipseg-rd64-refined for cloud segmentation on Sentinel-2 satellite imagery using the CloudSEN12+ dataset. This model is part of the research presented in:

Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift Harshith Kethavath, Weiming Hu EarthVision Workshop @ CVPR 2026

All models from this paper: https://huggingface.co/collections/uga-gaim/2026-cloudprompts

Model Description

CLIPSeg is a vision-language segmentation model trained on natural images. This variant is fully fine-tuned on CloudSEN12+ to adapt it to Sentinel-2 satellite imagery for four-class cloud segmentation: clear, thick cloud, thin cloud, and cloud shadow.

Developed by: Harshith Kethavath, Weiming Hu
Lab: Lab for Geoinformatics and AI Modeling (GAIM), University of Georgia
License: CC BY 4.0
Base model: CIDAS/clipseg-rd64-refined

How to Get Started

from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation
import torch
from PIL import Image

processor = CLIPSegProcessor.from_pretrained("uga-gaim/CLIPSeg-CloudSEN12Plus-FFT")
model = CLIPSegForImageSegmentation.from_pretrained("uga-gaim/CLIPSeg-CloudSEN12Plus-FFT")

image = Image.open("your_sentinel2_image.png")
prompts = ["clear", "thick cloud", "thin cloud", "cloud shadow"]

inputs = processor(
    text=prompts,
    images=[image] * len(prompts),
    return_tensors="pt",
    padding=True
)

with torch.no_grad():
    outputs = model(**inputs)

logits = outputs.logits  # shape: (4, H, W)
predicted_class = logits.argmax(dim=0)  # per-pixel class prediction

Training Details

Training Data

Trained on the CloudSEN12+ dataset, the largest expert-labeled cloud segmentation benchmark for Sentinel-2 imagery. 100% of the training split was used (full data setting).

Training Hyperparameters

Hyperparameter	Value
Optimizer	AdamW
Learning rate	5e-5
Weight decay	0.02
Warmup ratio	0.06
Epochs	20
Batch size	16
Precision	fp16

Loss Function

Combined segmentation loss: weighted sum of Focal loss, Tversky loss, and Boundary loss.

Evaluation Results

Evaluated on the CloudSEN12+ test split. Per-class IoU:

Class	Zero-Shot (baseline)	This model (FFT 100%)
Clear	0.5205	0.8540
Thick Cloud	0.2773	0.7875
Thin Cloud	0.0898	0.4702
Cloud Shadow	0.1325	0.5173
mIoU	0.2550	0.6572

Citation

@InProceedings{Kethavath_2026_CVPR,
    author    = {Kethavath, Harshith and Hu, Weiming},
    title     = {Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2026},
    pages     = {7960-7969}
}

Sources

1 source

Hugging FacePrincipalVersion 34ea01f8b42d0fdee389fb8e3b5fa5b7f53158e3

huggingface.co/uga-gaim/CLIPSeg-CloudSEN12Plus-FFT/tree/34ea01f8b42d0fdee389fb8e3b5fa5b7f53158e3

Vérifié 1 nov.

Download files

Repository files

File inventory only; completeness is unverified.

Artefacts du modèle

1 artefact

model.safetensors

safetensors · FP32 (unverified) · 575 MB · SHA-256 6e27178870b0…671d · Hugging Face

Download file

Extraits de sources

3 extraits

Readme18 aoûthuggingface.co/uga-gaim/CLIPSeg-CloudSEN12Plus-FFT

--- license: cc-by-4.0 datasets: - isp-uv-es/CloudSEN12Plus base_model: - CIDAS/clipseg-rd64-refined language: - en pipeline_tag: image-segmentation tags: - cloud - segmentation - remote-sensing - satellite - earth-observation --- # CLIPSeg Fine-tuned for Cloud Segmentation (Full Fine-Tuning, 100% Data) Fine-tuned version of [CIDAS/clipseg-rd64-refined](https://huggingface.co/CIDAS/clipseg-rd64-refined) for cloud segmentation on Sentinel-2 satellite imagery using the CloudSEN12+ dataset. This model is part of the research presented in: > **Low-Data Supervised Adaptation Outperforms Prompting for Cloud Segmentation Under Domain Shift** > Harshith Kethavath, Weiming Hu > EarthVision Workshop @ CVPR 2026 All models from this paper: [https://huggingface.co/collections/uga-gaim/2026-cloudprompts](https://huggingface.co/collections/uga-gaim/2026-cloudprompts) ## Model Description CLIPSeg is a vision-language segmentation model trained on natural images. This variant is fully fine-tuned on CloudSEN12+ to adapt it to Sentinel-2 satellite imagery for four-class cloud segmentation: clear, thick cloud, thin cloud, and cloud shadow. - **Developed by:** Harshith Kethavath, Weiming Hu - **Lab:** Lab for Geoinformatics and AI Modeling (GAIM), University of Georgia - **License:** CC BY 4.0 - **Base model:** CIDAS/clipseg-rd64-refined ## How to Get Started ```python from transformers import CLIPSegProcessor, CLIPSegForImageSegmentation import torch from PIL import Image processor = CLIPSegProcessor.from_pretrained("uga-gaim/CLIPSeg-CloudSEN12Plus-FFT") model = CLIPSegForImageSegmentation.from_pretrained("uga-gaim/CLIPSeg-CloudSEN12Plus-FFT") image = Image.open("your_sentinel2_image.png") prompts = ["clear", "thick cloud", "thin cloud", "cloud shadow"] inputs = processor( text=prompts, images=[image] * len(prompts), return_tensors="pt", padding=True ) with torch.no_grad(): outputs = model(**inputs) logits = outputs.logits # shape: (4, H, W) predicted_class = logits.argmax(dim=0) # per-pixel class prediction ``` ## Training Details ### Training Data Trained on the [CloudSEN12+](https://huggingface.co/datasets/isp-uv-es/CloudSEN12Plus) dataset, the largest expert-labeled cloud segmentation benchmark for Sentinel-2 imagery. 100% of the training split was used (full data setting). ### Training Hyperparameters | Hyperparameter | Value | |----------------...

Note1 nov.huggingface.co/uga-gaim/CLIPSeg-CloudSEN12Plus-FFT

Source context: 46 downloads · 0 likes · Pipeline image-segmentation · Repo uga-gaim/CLIPSeg-CloudSEN12Plus-FFT

Note18 aoûthuggingface.co/uga-gaim/CLIPSeg-CloudSEN12Plus-FFT

Source context: 26 downloads · 0 likes · Pipeline image-segmentation · Repo uga-gaim/CLIPSeg-CloudSEN12Plus-FFT