Thireus/mtp-Qwen3.5-397B-A17B-THIREUS-IQ2_K-SPECIAL_SPLIT · Hugging Face

--- license: mit --- # mtp-Qwen3.5-397B-A17B ## 🤔 What is this [HuggingFace repository](https://huggingface.co/Thireus/mtp-Qwen3.5-397B-A17B-THIREUS-BF16-SPECIAL_SPLIT/) about? This repository provides **GGUF-quantized tensors** for the [mtp](https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md) layer(s) of the Qwen3.5-397B-A17B model (official repo: https://huggingface.co/Qwen/Qwen3.5-397B-A17B). These GGUF shards are designed to be used with **Thireus’ GGUF Tool Suite** (https://github.com/Thireus/GGUF-Tool-Suite), a collection of tools that automatically finds the perplexity-optimal mix of quantizations for any given a model size target. With this GGUF Tool Suite, you can produce your own Dynamic 3.0 Quants recipes and achieve optimum accuracy & SOTA quantization performance. Give it a try here: https://gguf.thireus.com/quant_assign.html - 📖 Documentation: https://github.com/Thireus/GGUF-Tool-Suite/tree/main/docs - 🔍 Example of GGUF recipes: https://github.com/Thireus/GGUF-Tool-Suite/tree/main/recipe_examples - 🍳 Cook your own recipe files: https://gguf.thireus.com/quant_assign.html - ☁️ Download GGUF models from recipe files: https://gguf.thireus.com/quant_downloader.html - 📂 Browse available models: https://huggingface.co/Thireus/collections and https://gguf.thireus.com *tl;dr: Expand the details section below* <details> ``` cd ~ # Make sure to install all ik_llama.cpp compilation dependencies... apt install python3-dev python3-pip python3-venv python3-wheel python3-setuptools git acl netcat-openbsd cmake # pipx # Obtain ik_llama's Thireus version - Windows/macOS/Linux builds available at https://github.com/Thireus/ik_llama.cpp/releases git clone https://github.com/Thireus/ik_llama.cpp cd ik_llama.cpp git pull # Build ik_llama.cpp cmake -B build -DGGML_AVX=ON -DGGML_AVX2=ON -DLLAMA_CURL=OFF -DGGML_MAX_CONTEXTS=2048 cmake --build build --config Release -j16 cd .. # Obtain Thireus' GGUF-Tool-Suite GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/Thireus/GGUF-Tool-Suite # Download model quant mix from recipe file - you can also try the web version: https://gguf.thireus.com/quant_downloader.html cd GGUF-Tool-Suite rm -f download.conf # Make sure to copy the relevant download.conf for the model before running quant_assign.py cp -f models/Qwen3.5-397B-A17B/download.conf . # Use the download.conf of the chosen model mkdir -p...