Btoks Qwen3VL 2B Instruct
Btoks is a multimodal embedding model fine-tuned from Qwen/Qwen3-VL-2B-Instruct with the Bottleneck Tokens method.
Source du modèle
- Type
- Other
- Taille du fichier
- 264 MB–1,85 GB
- Modèle de base
- Qwen/Qwen3-VL-2B-Instruct
- Version
- vRevision 7552073
- Mis à jour
- 22 oct. 2026
- Téléchargements
- 6
- Source
- Hugging Face
Description de la source
Btoks Qwen3VL 2B Instruct
Btoks is a multimodal embedding model fine-tuned from
Qwen/Qwen3-VL-2B-Instruct with the Bottleneck Tokens method.
- Model name on MMEB-V2 leaderboard:
Btoks - Base model:
Qwen/Qwen3-VL-2B-Instruct - Paper: https://arxiv.org/abs/2604.11095
- Model repository:
https://huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct
MMEB-V2
Local validation with the public MMEB-V2 leaderboard scripts gives:
| Overall | Image-Overall | Video-Overall | Visdoc-Overall |
|---|---|---|---|
| 68.29 | 71.55 | 49.12 | 77.77 |
These numbers are not expected to match the paper tables one-to-one. After the paper experiments, we fixed a small number of data/evaluation bugs and trained with a larger data mixture and scale.
Training Summary
The model was trained with a compact BToks/SIEVE setup on top of Qwen3-VL:
- BToks / SIEVE tokens: 4
- LoRA rank / alpha / dropout: 16 / 32 / 0.05
- DoRA: enabled
- Generation-loss weight: 0.2
- Training steps: 5000
- Exported checkpoint: step 4500
- Contrastive temperature: 0.02
The main training data sources include:
- Image classification, VQA, retrieval, and grounding data, including ImageNet, N24News, VOC2007, SUN397, OK-VQA, A-OKVQA, DocVQA, ChartQA, Visual7W, GQA, TextVQA, VizWiz, VisDial, CIRR, VisualNews, MSCOCO, WebQA, FashionIQ, Wiki-SS-NQ, OVEN, EDIS, INFOSEEK, Fashion200K, and RefCOCO.
- Visual document retrieval data from the ColPali / ViDoRe family and VisRAG in-domain training data.
- Video classification, retrieval, QA, and moment-retrieval data, including Kinetics-700, Something-Something V2, HMDB51, UCF101, MSR-VTT, MSVD, DiDeMo, YouCook2, ActivityNet Captions, VATEX, QVHighlights, Charades-STA, NExTQA, and VideoChat2-IT.
Some source names overlap with MMEB-V2 benchmark dataset names because they come from the same public dataset families. The MMEB-V2 benchmark records were removed from the admitted training splits where overlap was possible, so the leaderboard evaluation does not use leaked benchmark samples for training.
Loading
This repository stores a merged inference checkpoint for the VLM2Emb/Btoks embedding wrapper. It is not a plain Qwen3-VL causal language model.
The public inference/evaluation code is being prepared and is expected to be released in about one week. Loading examples will be added once that code release is ready.
Sources
1 source
Sources
1 sourceVérifié 22 oct.
Download files
Artefacts du modèle
3 artefacts
Artefacts du modèle
3 artefactsmodel-00001-of-00003.safetensors
safetensors · 1,85 GB · SHA-256 c8e8d9e4f701…7e5c · Hugging Face
Download shardmodel-00002-of-00003.safetensors
safetensors · 1,85 GB · SHA-256 d778c4c7fb16…d38f · Hugging Face
Download shardmodel-00003-of-00003.safetensors
safetensors · 264 MB · SHA-256 c1e05f73323f…1f97 · Hugging Face
Download shardExtraits de sources
3 extraits
Extraits de sources
3 extraits--- library_name: transformers license: apache-2.0 base_model: Qwen/Qwen3-VL-2B-Instruct tags: - multimodal - embedding - retrieval - qwen3-vl - btoks pipeline_tag: image-feature-extraction --- # Btoks Qwen3VL 2B Instruct Btoks is a multimodal embedding model fine-tuned from `Qwen/Qwen3-VL-2B-Instruct` with the Bottleneck Tokens method. - Model name on MMEB-V2 leaderboard: `Btoks` - Base model: `Qwen/Qwen3-VL-2B-Instruct` - Paper: [https://arxiv.org/abs/2604.11095](https://arxiv.org/abs/2604.11095) - Model repository: `https://huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct` ## MMEB-V2 Local validation with the public MMEB-V2 leaderboard scripts gives: | Overall | Image-Overall | Video-Overall | Visdoc-Overall | | ---: | ---: | ---: | ---: | | 68.29 | 71.55 | 49.12 | 77.77 | These numbers are not expected to match the paper tables one-to-one. After the paper experiments, we fixed a small number of data/evaluation bugs and trained with a larger data mixture and scale. ## Training Summary The model was trained with a compact BToks/SIEVE setup on top of Qwen3-VL: - BToks / SIEVE tokens: 4 - LoRA rank / alpha / dropout: 16 / 32 / 0.05 - DoRA: enabled - Generation-loss weight: 0.2 - Training steps: 5000 - Exported checkpoint: step 4500 - Contrastive temperature: 0.02 The main training data sources include: - Image classification, VQA, retrieval, and grounding data, including ImageNet, N24News, VOC2007, SUN397, OK-VQA, A-OKVQA, DocVQA, ChartQA, Visual7W, GQA, TextVQA, VizWiz, VisDial, CIRR, VisualNews, MSCOCO, WebQA, FashionIQ, Wiki-SS-NQ, OVEN, EDIS, INFOSEEK, Fashion200K, and RefCOCO. - Visual document retrieval data from the ColPali / ViDoRe family and VisRAG in-domain training data. - Video classification, retrieval, QA, and moment-retrieval data, including Kinetics-700, Something-Something V2, HMDB51, UCF101, MSR-VTT, MSVD, DiDeMo, YouCook2, ActivityNet Captions, VATEX, QVHighlights, Charades-STA, NExTQA, and VideoChat2-IT. Some source names overlap with MMEB-V2 benchmark dataset names because they come from the same public dataset families. The MMEB-V2 benchmark records were removed from the admitted training splits where overlap was possible, so the leaderboard evaluation does not use leaked benchmark samples for training. ## Loading This repository stores a merged inference checkpoint for the VLM2Emb/Btoks embedding wrapper. It is...
Source context: 6 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo siyrus/Btoks-Qwen3VL-2B-Instruct
Source context: 13 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo siyrus/Btoks-Qwen3VL-2B-Instruct
