LibreSigLIP2so400m Cls

--- license: apache-2.0 library_name: libreyolo pipeline_tag: zero-shot-image-classification tags: - zero-shot-image-classification - siglip2 --- # LibreSigLIP2so400m-cls SigLIP 2 shape-optimized 400M patch-14 (384 px) dual-tower zero-shot image classifier, repackaged for LibreYOLO. The accuracy tier of the LibreSigLIP2 family. Open vocabulary: define your own class set at runtime with `set_classes`, no training needed. Multilingual (SentencePiece, Gemma vocabulary), with an optional `multi_label=True` sigmoid scoring mode. ## Usage ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSigLIP2so400m-cls.pt") model.set_classes(["a forklift", "an empty aisle", "a spill"]) r = model.predict("frame.jpg")[0] print(model.names[r.probs.top1], float(r.probs.top1conf)) ``` Requires the `libreyolo[siglip2]` extra (SentencePiece tokenizer). Note: this model runs at 384 px with a 27-layer 1152-wide ViT; it is heavy on CPU, prefer a GPU. ## Source Derived from [google/siglip2-so400m-patch14-384](https://huggingface.co/google/siglip2-so400m-patch14-384) at commit `e8e487298228002f3d8a82e0cd5c8ea9c567f57f`. Copyright (c) Google LLC. Licensed under the Apache License 2.0. Reference implementations: [google-research/big_vision](https://github.com/google-research/big_vision) and the Hugging Face `transformers` SigLIP code (both Apache-2.0). This fixed-resolution SigLIP 2 checkpoint uses the original SigLIP transformer architecture (`model_type: "siglip"`); the NaFlex variants are not included. ## Modifications State-dict metadata wrap only (LibreYOLO v1.0 checkpoint schema). Learned parameters are unchanged. See `weights/convert_siglip2_weights.py` in the [LibreYOLO source repository](https://github.com/LibreYOLO/libreyolo). Parity vs the reference `transformers` model at fp32 CPU: text tower and vision encoder are bit-identical (max_abs_diff == 0); the attention-pooled image feature differs by < 1e-6 (a torch MultiheadAttention fused-kernel residual, documented in `weights/parity_siglip2.py`). ## Zero-shot accuracy Imagenette (10-class, val, `model.val()` zero-shot, prompt "a photo of a {}."): 99.5 top-1 (160 px source images), 99.6 top-1 (320 px source images). Beats LibreSigLIP2b16-cls on both. ## License Apache License 2.0. See the [`LICENSE`](./LICENSE) and [`NOTICE`](./NOTICE) files in this repository. Weights were trained by Google on t...

Source context: 0 downloads · 0 likes · Pipeline zero-shot-image-classification · Library libreyolo · Repo LibreYOLO/LibreSigLIP2so400m-cls