VisionEncoder

--- license: apache-2.0 library_name: transformers tags: - vision-encoder - distillation - video-language - siglip2 - dinov3 --- # VisionEncoder Hosted artifacts (derived data + trained checkpoints) for the **VisionEncoder** research project. **Training code + full reproduction guide**: https://github.com/xiaomoguhz/VisionEncoder The repo is organized into three top-level folders. ## `data/` — current (V9.x) reproduction data (~6.5G) | Path | Content | |---|---| | `data/vmllm_cached/qwen3vit/` | S2 `cached_dataset` arrow (image/video, 10pct + full); fed directly to stage-2 | | `data/ms-swift-data/` | sampled sharegpt jsonl (10pct + full) | | `data/llava_video/` | V9 decode-probed `good_manifest` for the video path | ## `ckpts/` — ready-made 4B MLLM inference weights | Path | Content | |---|---| | `ckpts/4b_stock` | 4B stock baseline (raw Qwen3.5 ViT, skips declip), checkpoint-505, 9.5G | | `ckpts/4b_v9_1` | 4B V9.1 (V-JEPA 2.1 video self-distill), checkpoint-505, 9.5G | Download either and feed it straight to evaluation (see the GitHub README, section 4 — MLLM evaluation) to skip declip + S1 + S2. ## `legacy/` — historical assets (~368G) Early-line products, not needed to reproduce the current main line: `declip_siglip2/spatial_align`, `kd_mllm`, `self_refine`, `video_mllm_swift` (old SigLIP2 / image-only S1+S2 ckpts), and old ViT-family arrow caches. ## Download ```bash # current dev data huggingface-cli download xiaomoguhzz/VisionEncoder --include "data/*" --local-dir . # ready-made 4B MLLM ckpt (eval directly) huggingface-cli download xiaomoguhzz/VisionEncoder --include "ckpts/4b_v9_1/*" --local-dir . ``` ## Related - Code + reproduction guide: https://github.com/xiaomoguhz/VisionEncoder