Btoks Qwen3VL 2B Instruct

--- library_name: transformers license: apache-2.0 base_model: Qwen/Qwen3-VL-2B-Instruct tags: - multimodal - embedding - retrieval - qwen3-vl - btoks pipeline_tag: image-feature-extraction --- # Btoks Qwen3VL 2B Instruct Btoks is a multimodal embedding model fine-tuned from `Qwen/Qwen3-VL-2B-Instruct` with the Bottleneck Tokens method. - Model name on MMEB-V2 leaderboard: `Btoks` - Base model: `Qwen/Qwen3-VL-2B-Instruct` - Paper: [https://arxiv.org/abs/2604.11095](https://arxiv.org/abs/2604.11095) - Model repository: `https://huggingface.co/siyrus/Btoks-Qwen3VL-2B-Instruct` ## MMEB-V2 Local validation with the public MMEB-V2 leaderboard scripts gives: | Overall | Image-Overall | Video-Overall | Visdoc-Overall | | ---: | ---: | ---: | ---: | | 68.29 | 71.55 | 49.12 | 77.77 | These numbers are not expected to match the paper tables one-to-one. After the paper experiments, we fixed a small number of data/evaluation bugs and trained with a larger data mixture and scale. ## Training Summary The model was trained with a compact BToks/SIEVE setup on top of Qwen3-VL: - BToks / SIEVE tokens: 4 - LoRA rank / alpha / dropout: 16 / 32 / 0.05 - DoRA: enabled - Generation-loss weight: 0.2 - Training steps: 5000 - Exported checkpoint: step 4500 - Contrastive temperature: 0.02 The main training data sources include: - Image classification, VQA, retrieval, and grounding data, including ImageNet, N24News, VOC2007, SUN397, OK-VQA, A-OKVQA, DocVQA, ChartQA, Visual7W, GQA, TextVQA, VizWiz, VisDial, CIRR, VisualNews, MSCOCO, WebQA, FashionIQ, Wiki-SS-NQ, OVEN, EDIS, INFOSEEK, Fashion200K, and RefCOCO. - Visual document retrieval data from the ColPali / ViDoRe family and VisRAG in-domain training data. - Video classification, retrieval, QA, and moment-retrieval data, including Kinetics-700, Something-Something V2, HMDB51, UCF101, MSR-VTT, MSVD, DiDeMo, YouCook2, ActivityNet Captions, VATEX, QVHighlights, Charades-STA, NExTQA, and VideoChat2-IT. Some source names overlap with MMEB-V2 benchmark dataset names because they come from the same public dataset families. The MMEB-V2 benchmark records were removed from the admitted training splits where overlap was possible, so the leaderboard evaluation does not use leaked benchmark samples for training. ## Loading This repository stores a merged inference checkpoint for the VLM2Emb/Btoks embedding wrapper. It is...

Source context: 6 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo siyrus/Btoks-Qwen3VL-2B-Instruct

Source context: 13 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo siyrus/Btoks-Qwen3VL-2B-Instruct