Metaclip 2 Vit Giant Patch14 224
MetaCLIP-2 is a family of state-of-the-art vision-language models developed by Meta AI. It scales the Metadata-Curated (MetaCLIP) approach to significantly larger datasets and more powerful architectures. By...
Source du modèle
- Type
- Embedding
- Taille du fichier
- 6,66–6,87 GB
- Version
- vRevision 73915b9
- Mis à jour
- 2 oct. 2026
- Éditeur
- @keras
- Téléchargements
- 3
- Source
- Hugging Face
Description de la source
Model Overview
Model Summary
MetaCLIP-2 is a family of state-of-the-art vision-language models developed by Meta AI. It scales the Metadata-Curated (MetaCLIP) approach to significantly larger datasets and more powerful architectures. By utilizing an automated curation pipeline that matches web-scale data to the distribution of high-quality datasets, MetaCLIP-2 achieves industry-leading performance in zero-shot classification and image-text retrieval without the need for proprietary datasets.
Links
- MetaCLIP2 Technical Paper
- MetaCLIP2 API Documentation
- MetaCLIP2 Inference Tutorial
- KerasHub Beginner Guide
- KerasHub Model Publishing Guide
Usage
MetaCLIP-2 can be used with Keras Hub to extract embeddings for images and text or to perform zero-shot classification.
Installation
pip install --upgrade keras-hub
pip install --upgrade keras
Presets
The following presets are available in Keras Hub. These presets incorporate the Metadata-Curated training methodology at various scales of the Vision Transformer (ViT) architecture.
| Preset Name | Description | Parameters | Input Resolution |
|---|---|---|---|
metaclip_2_vit_huge_patch14_224 | ViT-H/14 backbone trained on 224x224 images using the worldwide dataset. | ~1.1B | 224x224 |
metaclip_2_vit_huge_patch14_378 | ViT-H/14 backbone fine-tuned on 378x378 resolution for high-detail tasks. | ~1.1B | 378x378 |
metaclip_2_vit_giant_patch14_224 | ViT-bigG/14 backbone, the largest MetaCLIP-2 variant, trained on 224x224 images. | ~2.5B | 224x224 |
metaclip_2_vit_giant_patch14_378 | ViT-bigG/14 backbone fine-tuned on 378x378 resolution for maximum performance. | ~2.5B | 378x378 |
Model Architecture
MetaCLIP-2 utilizes a dual-encoder architecture consisting of a Vision Transformer (ViT) and a Text Transformer.
- Vision Encoder: Depending on the preset, it uses either a "Huge" (ViT-H) or "Giant" (ViT-bigG) architecture. These models process images as a sequence of patches (14x14).
- Text Encoder: A standard Transformer architecture that tokenizes and embeds textual descriptions into the same latent space as the images.
- Training Objective: The model is trained using a contrastive loss (InfoNCE), which maximizes the cosine similarity between matching image-text pairs while minimizing it for non-matching pairs.
The defining characteristic of MetaCLIP-2 is its Data Pipeline. Instead of relying on raw web crawls, Meta uses a metadata-curated approach that filters and balances billions of image-text pairs to ensure the training signal is representative of high-quality visual concepts.
License
MetaCLIP-2 is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. For more details, please refer to the official Meta AI repository.
Sources
1 source
Sources
1 sourceVérifié 2 oct.
Artefacts du modèle
2 artefacts
Artefacts du modèle
2 artefactsExtraits de sources
3 extraits
Extraits de sources
3 extraits--- library_name: keras-hub --- ### Model Overview # Model Summary MetaCLIP-2 is a family of state-of-the-art vision-language models developed by Meta AI. It scales the Metadata-Curated (MetaCLIP) approach to significantly larger datasets and more powerful architectures. By utilizing an automated curation pipeline that matches web-scale data to the distribution of high-quality datasets, MetaCLIP-2 achieves industry-leading performance in zero-shot classification and image-text retrieval without the need for proprietary datasets. ## Links * [MetaCLIP2 Technical Paper](https://arxiv.org/abs/2507.22062) * [MetaCLIP2 API Documentation]() * [MetaCLIP2 Inference Tutorial](https://www.kaggle.com/code/sravanneeli/metaclip2-inference-tutorial) * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/) * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/) ## Usage MetaCLIP-2 can be used with Keras Hub to extract embeddings for images and text or to perform zero-shot classification. ### Installation ```bash pip install --upgrade keras-hub pip install --upgrade keras ``` ## Presets The following presets are available in Keras Hub. These presets incorporate the Metadata-Curated training methodology at various scales of the Vision Transformer (ViT) architecture. | Preset Name | Description | Parameters | Input Resolution | | :--- | :--- | :--- | :--- | | `metaclip_2_vit_huge_patch14_224` | ViT-H/14 backbone trained on 224x224 images using the worldwide dataset. | ~1.1B | 224x224 | | `metaclip_2_vit_huge_patch14_378` | ViT-H/14 backbone fine-tuned on 378x378 resolution for high-detail tasks. | ~1.1B | 378x378 | | `metaclip_2_vit_giant_patch14_224` | ViT-bigG/14 backbone, the largest MetaCLIP-2 variant, trained on 224x224 images. | ~2.5B | 224x224 | | `metaclip_2_vit_giant_patch14_378` | ViT-bigG/14 backbone fine-tuned on 378x378 resolution for maximum performance. | ~2.5B | 378x378 | ## Model Architecture MetaCLIP-2 utilizes a dual-encoder architecture consisting of a **Vision Transformer (ViT)** and a **Text Transformer**. - **Vision Encoder:** Depending on the preset, it uses either a "Huge" (ViT-H) or "Giant" (ViT-bigG) architecture. These models process images as a sequence of patches (14x14). - **Text Encoder:** A standard Transformer architecture that tokenizes and embeds textual descriptions into the same...
Source context: 3 downloads · 0 likes · Library keras-hub · Repo keras/metaclip_2_vit_giant_patch14_224
Source context: 2 downloads · 0 likes · Library keras-hub · Repo keras/metaclip_2_vit_giant_patch14_224
