Skip to content
ModelListedExternal
Browse files

Btoks Qwen3VL 2B Instruct

Btoks is a multimodal embedding model fine-tuned from Qwen/Qwen3-VL-2B-Instruct with the Bottleneck Tokens method.

Sourcehuggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct/tree/7552073fd32764dff7b09982fca39a2d6b4582d5
KindOtherBase modelQwen/Qwen3-VL-2B-InstructVersionRevision 7552073Bgrade

Model source

Kind
Other
File size
264 MB–1.85 GB
Base model
Qwen/Qwen3-VL-2B-Instruct
Version
vRevision 7552073
Updated
Oct 22, 2026
Downloads
6
Source
Hugging Face

Source description

Btoks Qwen3VL 2B Instruct

Btoks is a multimodal embedding model fine-tuned from Qwen/Qwen3-VL-2B-Instruct with the Bottleneck Tokens method.

  • Model name on MMEB-V2 leaderboard: Btoks
  • Base model: Qwen/Qwen3-VL-2B-Instruct
  • Paper: https://arxiv.org/abs/2604.11095
  • Model repository: https://huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct

MMEB-V2

Local validation with the public MMEB-V2 leaderboard scripts gives:

OverallImage-OverallVideo-OverallVisdoc-Overall
68.2971.5549.1277.77

These numbers are not expected to match the paper tables one-to-one. After the paper experiments, we fixed a small number of data/evaluation bugs and trained with a larger data mixture and scale.

Training Summary

The model was trained with a compact BToks/SIEVE setup on top of Qwen3-VL:

  • BToks / SIEVE tokens: 4
  • LoRA rank / alpha / dropout: 16 / 32 / 0.05
  • DoRA: enabled
  • Generation-loss weight: 0.2
  • Training steps: 5000
  • Exported checkpoint: step 4500
  • Contrastive temperature: 0.02

The main training data sources include:

  • Image classification, VQA, retrieval, and grounding data, including ImageNet, N24News, VOC2007, SUN397, OK-VQA, A-OKVQA, DocVQA, ChartQA, Visual7W, GQA, TextVQA, VizWiz, VisDial, CIRR, VisualNews, MSCOCO, WebQA, FashionIQ, Wiki-SS-NQ, OVEN, EDIS, INFOSEEK, Fashion200K, and RefCOCO.
  • Visual document retrieval data from the ColPali / ViDoRe family and VisRAG in-domain training data.
  • Video classification, retrieval, QA, and moment-retrieval data, including Kinetics-700, Something-Something V2, HMDB51, UCF101, MSR-VTT, MSVD, DiDeMo, YouCook2, ActivityNet Captions, VATEX, QVHighlights, Charades-STA, NExTQA, and VideoChat2-IT.

Some source names overlap with MMEB-V2 benchmark dataset names because they come from the same public dataset families. The MMEB-V2 benchmark records were removed from the admitted training splits where overlap was possible, so the leaderboard evaluation does not use leaked benchmark samples for training.

Loading

This repository stores a merged inference checkpoint for the VLM2Emb/Btoks embedding wrapper. It is not a plain Qwen3-VL causal language model.

The public inference/evaluation code is being prepared and is expected to be released in about one week. Loading examples will be added once that code release is ready.

Sources

1 source
Hugging FacePrimaryVersion 7552073fd32764dff7b09982fca39a2d6b4582d5
huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct/tree/7552073fd32764dff7b09982fca39a2d6b4582d5

Verified Oct 22

Download files

Model artifacts

3 artifacts

model-00001-of-00003.safetensors

safetensors · 1.85 GB · SHA-256 c8e8d9e4f701…7e5c · Hugging Face

Download shard

model-00002-of-00003.safetensors

safetensors · 1.85 GB · SHA-256 d778c4c7fb16…d38f · Hugging Face

Download shard

model-00003-of-00003.safetensors

safetensors · 264 MB · SHA-256 c1e05f73323f…1f97 · Hugging Face

Download shard

Source excerpts

3 excerpts
ReadmeAug 5huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct

--- library_name: transformers license: apache-2.0 base_model: Qwen/Qwen3-VL-2B-Instruct tags: - multimodal - embedding - retrieval - qwen3-vl - btoks pipeline_tag: image-feature-extraction --- # Btoks Qwen3VL 2B Instruct Btoks is a multimodal embedding model fine-tuned from `Qwen/Qwen3-VL-2B-Instruct` with the Bottleneck Tokens method. - Model name on MMEB-V2 leaderboard: `Btoks` - Base model: `Qwen/Qwen3-VL-2B-Instruct` - Paper: [https://arxiv.org/abs/2604.11095](https://arxiv.org/abs/2604.11095) - Model repository: `https://huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct` ## MMEB-V2 Local validation with the public MMEB-V2 leaderboard scripts gives: | Overall | Image-Overall | Video-Overall | Visdoc-Overall | | ---: | ---: | ---: | ---: | | 68.29 | 71.55 | 49.12 | 77.77 | These numbers are not expected to match the paper tables one-to-one. After the paper experiments, we fixed a small number of data/evaluation bugs and trained with a larger data mixture and scale. ## Training Summary The model was trained with a compact BToks/SIEVE setup on top of Qwen3-VL: - BToks / SIEVE tokens: 4 - LoRA rank / alpha / dropout: 16 / 32 / 0.05 - DoRA: enabled - Generation-loss weight: 0.2 - Training steps: 5000 - Exported checkpoint: step 4500 - Contrastive temperature: 0.02 The main training data sources include: - Image classification, VQA, retrieval, and grounding data, including ImageNet, N24News, VOC2007, SUN397, OK-VQA, A-OKVQA, DocVQA, ChartQA, Visual7W, GQA, TextVQA, VizWiz, VisDial, CIRR, VisualNews, MSCOCO, WebQA, FashionIQ, Wiki-SS-NQ, OVEN, EDIS, INFOSEEK, Fashion200K, and RefCOCO. - Visual document retrieval data from the ColPali / ViDoRe family and VisRAG in-domain training data. - Video classification, retrieval, QA, and moment-retrieval data, including Kinetics-700, Something-Something V2, HMDB51, UCF101, MSR-VTT, MSVD, DiDeMo, YouCook2, ActivityNet Captions, VATEX, QVHighlights, Charades-STA, NExTQA, and VideoChat2-IT. Some source names overlap with MMEB-V2 benchmark dataset names because they come from the same public dataset families. The MMEB-V2 benchmark records were removed from the admitted training splits where overlap was possible, so the leaderboard evaluation does not use leaked benchmark samples for training. ## Loading This repository stores a merged inference checkpoint for the VLM2Emb/Btoks embedding wrapper. It is...

NoteOct 22huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct

Source context: 6 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo siyrus/Btoks-Qwen3VL-2B-Instruct

NoteAug 5huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct

Source context: 13 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo siyrus/Btoks-Qwen3VL-2B-Instruct