Qwen AgentWorld 35B A3B UD Q4 K XL MTP GGUF

--- license: apache-2.0 base_model: - Qwen/Qwen-AgentWorld-35B-A3B pipeline_tag: image-text-to-text tags: - moe - multimodal - vision - mtp - gguf --- # Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF This repository contains the Qwen-AgentWorld-35B-A3B model in UD-Q4_K_XL quantization with Multi-Token Prediction (MTP) support for speculative decoding. ## Acknowledgments - **Qwen Team** for the original [Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B) model - **Unsloth** for the [Qwen3.6-35B-A3B-MTP-GGUF](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF) quantization with MTP heads - **Unsloth** for the [Qwen-AgentWorld-35B-A3B-GGUF](https://huggingface.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF) quantization ## How It Was Made The MTP heads from Qwen3.6-35B-A3B were extracted and grafted onto the Qwen-AgentWorld-35B-A3B model. Both models share the same architecture (qwen35moe), making this transfer possible. The resulting model includes all base layers from AgentWorld plus the MTP prediction head from the original Qwen3.6 model. ## Hardware & Performance Performance results were achieved on **AMD Strix Halo**: - Without MTP: ~60 tokens/s - With MTP: ~77 tokens/s - **Speed improvement: +28%** - Draft acceptance rate: ~87% The high acceptance rate indicates that the MTP heads from the base model transfer well to the AgentWorld fine-tune. ## Usage ### Basic Usage ```bash ./llama-server \ -m Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf \ -ngl 99 -c 8192 -fa on \ --spec-type draft-mtp \ --spec-draft-n-max 3 ``` ### Full Multimodal Setup For multimodal capabilities with reasoning and optimized parameters: ```bash MODEL_PATH="Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf" MMPROJ_PATH="mmproj-BF16.gguf" ./llama-server \ --model "$MODEL_PATH" \ --mmproj "$MMPROJ_PATH" \ -ngl 99 \ --host 0.0.0.0 \ --port 8000 \ --ctx-size 262144 \ --reasoning "on" \ --kv-unified \ -np 2 \ -fa "on" \ --no-mmap \ --image-min-tokens 1024 \ --spec-type "draft-mtp" \ --spec-draft-n-max 3 \ --spec-draft-p-min 0.75 \ -b 2048 \ -ub 512 ``` **Parameters:** - `--mmproj`: Multimodal projector for vision capabilities - `--reasoning "on"`: Enable reasoning mode - `--kv-unified`: Unified KV cache - `-np 2`: 2 parallel slots - `--spec-type "draft-mtp"`: Enable MTP speculative decoding - `--spec-draft-n-max 3`: Generate up to 3 dra...

Source context: 482 downloads · 5 likes · Pipeline image-text-to-text · Repo vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF

Source context: 703 downloads · 4 likes · Pipeline image-text-to-text · Repo vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF