Metaclip 2 Vit Huge Patch14 378
MetaCLIP-2 is a family of state-of-the-art vision-language models developed by Meta AI. It scales the Metadata-Curated (MetaCLIP) approach to significantly larger datasets and more powerful architectures. By...
Fonte do modelo
- Tipo
- Other
- Tamanho do arquivo
- 6,93 GB
- Versão
- vRevision 1db8827
- Atualizado
- 2 de out. de 2026
- Publicador
- @keras
- Downloads
- 3
- Fonte
- Hugging Face
Descrição da fonte
Model Overview
Model Summary
MetaCLIP-2 is a family of state-of-the-art vision-language models developed by Meta AI. It scales the Metadata-Curated (MetaCLIP) approach to significantly larger datasets and more powerful architectures. By utilizing an automated curation pipeline that matches web-scale data to the distribution of high-quality datasets, MetaCLIP-2 achieves industry-leading performance in zero-shot classification and image-text retrieval without the need for proprietary datasets.
Links
- MetaCLIP2 Technical Paper
- MetaCLIP2 API Documentation
- MetaCLIP2 Inference Tutorial
- KerasHub Beginner Guide
- KerasHub Model Publishing Guide
Usage
MetaCLIP-2 can be used with Keras Hub to extract embeddings for images and text or to perform zero-shot classification.
Installation
pip install --upgrade keras-hub
pip install --upgrade keras
Presets
The following presets are available in Keras Hub. These presets incorporate the Metadata-Curated training methodology at various scales of the Vision Transformer (ViT) architecture.
| Preset Name | Description | Parameters | Input Resolution |
|---|---|---|---|
metaclip_2_vit_huge_patch14_224 | ViT-H/14 backbone trained on 224x224 images using the worldwide dataset. | ~1.1B | 224x224 |
metaclip_2_vit_huge_patch14_378 | ViT-H/14 backbone fine-tuned on 378x378 resolution for high-detail tasks. | ~1.1B | 378x378 |
metaclip_2_vit_giant_patch14_224 | ViT-bigG/14 backbone, the largest MetaCLIP-2 variant, trained on 224x224 images. | ~2.5B | 224x224 |
metaclip_2_vit_giant_patch14_378 | ViT-bigG/14 backbone fine-tuned on 378x378 resolution for maximum performance. | ~2.5B | 378x378 |
Model Architecture
MetaCLIP-2 utilizes a dual-encoder architecture consisting of a Vision Transformer (ViT) and a Text Transformer.
- Vision Encoder: Depending on the preset, it uses either a "Huge" (ViT-H) or "Giant" (ViT-bigG) architecture. These models process images as a sequence of patches (14x14).
- Text Encoder: A standard Transformer architecture that tokenizes and embeds textual descriptions into the same latent space as the images.
- Training Objective: The model is trained using a contrastive loss (InfoNCE), which maximizes the cosine similarity between matching image-text pairs while minimizing it for non-matching pairs.
The defining characteristic of MetaCLIP-2 is its Data Pipeline. Instead of relying on raw web crawls, Meta uses a metadata-curated approach that filters and balances billions of image-text pairs to ensure the training signal is representative of high-quality visual concepts.
License
MetaCLIP-2 is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. For more details, please refer to the official Meta AI repository.
Fontes
1 fonte
Fontes
1 fonteVerificado 2 de out.
Artefatos de modelo
1 artefato
Artefatos de modelo
1 artefatoTrechos de fonte
3 trechos
Trechos de fonte
3 trechos--- library_name: keras-hub --- ### Model Overview # Model Summary MetaCLIP-2 is a family of state-of-the-art vision-language models developed by Meta AI. It scales the Metadata-Curated (MetaCLIP) approach to significantly larger datasets and more powerful architectures. By utilizing an automated curation pipeline that matches web-scale data to the distribution of high-quality datasets, MetaCLIP-2 achieves industry-leading performance in zero-shot classification and image-text retrieval without the need for proprietary datasets. ## Links * [MetaCLIP2 Technical Paper](https://arxiv.org/abs/2507.22062) * [MetaCLIP2 API Documentation]() * [MetaCLIP2 Inference Tutorial](https://www.kaggle.com/code/sravanneeli/metaclip2-inference-tutorial) * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/) * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/) ## Usage MetaCLIP-2 can be used with Keras Hub to extract embeddings for images and text or to perform zero-shot classification. ### Installation ```bash pip install --upgrade keras-hub pip install --upgrade keras ``` ## Presets The following presets are available in Keras Hub. These presets incorporate the Metadata-Curated training methodology at various scales of the Vision Transformer (ViT) architecture. | Preset Name | Description | Parameters | Input Resolution | | :--- | :--- | :--- | :--- | | `metaclip_2_vit_huge_patch14_224` | ViT-H/14 backbone trained on 224x224 images using the worldwide dataset. | ~1.1B | 224x224 | | `metaclip_2_vit_huge_patch14_378` | ViT-H/14 backbone fine-tuned on 378x378 resolution for high-detail tasks. | ~1.1B | 378x378 | | `metaclip_2_vit_giant_patch14_224` | ViT-bigG/14 backbone, the largest MetaCLIP-2 variant, trained on 224x224 images. | ~2.5B | 224x224 | | `metaclip_2_vit_giant_patch14_378` | ViT-bigG/14 backbone fine-tuned on 378x378 resolution for maximum performance. | ~2.5B | 378x378 | ## Model Architecture MetaCLIP-2 utilizes a dual-encoder architecture consisting of a **Vision Transformer (ViT)** and a **Text Transformer**. - **Vision Encoder:** Depending on the preset, it uses either a "Huge" (ViT-H) or "Giant" (ViT-bigG) architecture. These models process images as a sequence of patches (14x14). - **Text Encoder:** A standard Transformer architecture that tokenizes and embeds textual descriptions into the same...
Source context: 3 downloads · 0 likes · Library keras-hub · Repo keras/metaclip_2_vit_huge_patch14_378
Source context: 1 downloads · 0 likes · Library keras-hub · Repo keras/metaclip_2_vit_huge_patch14_378
