--- license: mit base_model: - WeiboAI/VibeThinker-3B - Qwen/Qwen2.5-3B - Qwen/Qwen2.5-Coder-3B --- # VibeThinker-3B-GGUF ## Direct GGUF Quantizations of VibeThinker-3B This repository provides GGUF quantized models for **WeiboAI/VibeThinker-3B**. VibeThinker-3B is a powerful 3 billion parameter Small Language Model developed by WeiboAI. Built on the Qwen2.5 architecture, it is fine-tuned for challenging reasoning tasks with clear verification signals, excelling in mathematics, coding, and STEM reasoning. It achieves frontier-level performance on benchmarks like AIME, HMMT, IMO-AnswerBench, and LiveCodeBench, reaching a **96.1% acceptance rate** on recent LeetCode weekly/biweekly contests. These GGUF versions are optimized for efficient CPU and GPU inference using `llama.cpp` and compatible tools. This release includes various quantization levels (e.g., Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0) to suit different hardware capabilities and performance requirements. ## Table of Contents 📝 1. ▶ [Usage](#usage) 2. 📃 [License](#license) 3. 🙏 [Acknowledgements](#acknowledgements) ## ▶ Usage ### 1. Download Models Download models using `huggingface-cli`: ```bash pip install "huggingface_hub[cli]" huggingface-cli download samuelchristlie/VibeThinker-3B-gguf --local-dir ./VibeThinker-3B-gguf ``` You can also download directly from [this page](https://huggingface.co/samuelchristlie/VibeThinker-3B-gguf/tree/main) ### 2. Inference To use these GGUF files, you'll need a compatible inference engine like `llama.cpp` or clients built on top of it (e.g., `Ollama`, `LM Studio`, `KoboldCpp`, `text-generation-webui` with `llama.cpp` backend). > **Note:** VibeThinker-3B was not trained on tool-calling or agent-based programming data. It is best suited for competitive-style math, coding (e.g., LeetCode-style problems), and STEM reasoning tasks. For harder math reasoning, try evaluating against [AMOBench](https://huggingface.co/datasets/meituan-longcat/AMO-Bench) with `max_tokens` set to 60K–100K. ## 📃 License This model is a GGUF conversion of the original `WeiboAI/VibeThinker-3B` model. The original model is licensed under the **MIT License**, and this derivative work adheres to the terms of that license. Please review the original license for full details. ## 🙏 Acknowledgements * **WeiboAI** for developing and open-sourcing the powerful VibeThinker-3B model: * [VibeThinker-3B on Hugging Face](https://huggingface.co/WeiboAI/VibeThinker-3B) * [Technical Report (arXiv:2606.16140)](https://arxiv.org/abs/2606.16140) * [GitHub Repository](https://github.com/WeiboAI/VibeThinker) * The **llama.cpp** project and its contributors for the GGUF format and the incredible tooling that makes local LLM inference accessible. * [llama.cpp GitHub Repository](https://github.com/ggerganov/llama.cpp)
