ModelPublicadoExterno

Qwen AgentWorld 35B A3B UD Q4 K XL MTP GGUF

This repository contains the Qwen-AgentWorld-35B-A3B model in UD-Q4KXL quantization with Multi-Token Prediction (MTP) support for speculative decoding.

Fuentehuggingface.co/vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF/tree/882eb9cce9da627a771dd4804cff5a1359b97550

TipoModeloModelo baseQwen/Qwen-AgentWorld-35B-A3BVersiónRevision 882eb9cBcalidad

Fuente del modelo

Tipo: Modelo
Tamaño del archivo: 861 MB–25,3 GB
Modelo base: Qwen/Qwen-AgentWorld-35B-A3B
Versión: vRevision 882eb9c
Actualizado: 30 oct 2026
Descargas: 482
Fuente: Hugging Face

Descripción de la fuente

Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF

This repository contains the Qwen-AgentWorld-35B-A3B model in UD-Q4_K_XL quantization with Multi-Token Prediction (MTP) support for speculative decoding.

Acknowledgments

Qwen Team for the original Qwen-AgentWorld-35B-A3B model
Unsloth for the Qwen3.6-35B-A3B-MTP-GGUF quantization with MTP heads
Unsloth for the Qwen-AgentWorld-35B-A3B-GGUF quantization

How It Was Made

The MTP heads from Qwen3.6-35B-A3B were extracted and grafted onto the Qwen-AgentWorld-35B-A3B model. Both models share the same architecture (qwen35moe), making this transfer possible. The resulting model includes all base layers from AgentWorld plus the MTP prediction head from the original Qwen3.6 model.

Hardware & Performance

Performance results were achieved on AMD Strix Halo:

Without MTP: ~60 tokens/s
With MTP: ~77 tokens/s
Speed improvement: +28%
Draft acceptance rate: ~87%

The high acceptance rate indicates that the MTP heads from the base model transfer well to the AgentWorld fine-tune.

Usage

Basic Usage

./llama-server \
  -m Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf \
  -ngl 99 -c 8192 -fa on \
  --spec-type draft-mtp \
  --spec-draft-n-max 3

Full Multimodal Setup

For multimodal capabilities with reasoning and optimized parameters:

MODEL_PATH="Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf"
MMPROJ_PATH="mmproj-BF16.gguf"

./llama-server \
  --model "$MODEL_PATH" \
  --mmproj "$MMPROJ_PATH" \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8000 \
  --ctx-size 262144 \
  --reasoning "on" \
  --kv-unified \
  -np 2 \
  -fa "on" \
  --no-mmap \
  --image-min-tokens 1024 \
  --spec-type "draft-mtp" \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.75 \
  -b 2048 \
  -ub 512

Parameters:

--mmproj: Multimodal projector for vision capabilities
--reasoning "on": Enable reasoning mode
--kv-unified: Unified KV cache
-np 2: 2 parallel slots
--spec-type "draft-mtp": Enable MTP speculative decoding
--spec-draft-n-max 3: Generate up to 3 draft tokens per step
--spec-draft-p-min 0.75: Minimum probability threshold for draft tokens
-b 2048: Batch size
-ub 512: U-batch size

Fuentes

1 fuente

Hugging FacePrincipalVersión 882eb9cce9da627a771dd4804cff5a1359b97550

huggingface.co/vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF/tree/882eb9cce9da627a771dd4804cff5a1359b97550

Verificado 30 oct

Download files

Repository files

File inventory only; completeness is unverified.

Artefactos del modelo

3 artefactos

mmproj-BF16.gguf

gguf · BF16 (unverified) · 861 MB · SHA-256 da63cb47a767…5cfa · Hugging Face

Download file

Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf

gguf · Q4_K_XL (unverified) · 21,3 GB · SHA-256 dfc8aa15b34e…171e · Hugging Face

Download file

Qwen-AgentWorld-35B-A3B-UD-Q5_K_XL-MTP.gguf

gguf · Q5_K_XL (unverified) · 25,3 GB · SHA-256 e10b24965e0a…018a · Hugging Face

Download file

Extractos de fuentes

3 extractos

Readme14 agohuggingface.co/vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF

--- license: apache-2.0 base_model: - Qwen/Qwen-AgentWorld-35B-A3B pipeline_tag: image-text-to-text tags: - moe - multimodal - vision - mtp - gguf --- # Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF This repository contains the Qwen-AgentWorld-35B-A3B model in UD-Q4_K_XL quantization with Multi-Token Prediction (MTP) support for speculative decoding. ## Acknowledgments - **Qwen Team** for the original [Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B) model - **Unsloth** for the [Qwen3.6-35B-A3B-MTP-GGUF](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF) quantization with MTP heads - **Unsloth** for the [Qwen-AgentWorld-35B-A3B-GGUF](https://huggingface.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF) quantization ## How It Was Made The MTP heads from Qwen3.6-35B-A3B were extracted and grafted onto the Qwen-AgentWorld-35B-A3B model. Both models share the same architecture (qwen35moe), making this transfer possible. The resulting model includes all base layers from AgentWorld plus the MTP prediction head from the original Qwen3.6 model. ## Hardware & Performance Performance results were achieved on **AMD Strix Halo**: - Without MTP: ~60 tokens/s - With MTP: ~77 tokens/s - **Speed improvement: +28%** - Draft acceptance rate: ~87% The high acceptance rate indicates that the MTP heads from the base model transfer well to the AgentWorld fine-tune. ## Usage ### Basic Usage ```bash ./llama-server \ -m Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf \ -ngl 99 -c 8192 -fa on \ --spec-type draft-mtp \ --spec-draft-n-max 3 ``` ### Full Multimodal Setup For multimodal capabilities with reasoning and optimized parameters: ```bash MODEL_PATH="Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf" MMPROJ_PATH="mmproj-BF16.gguf" ./llama-server \ --model "$MODEL_PATH" \ --mmproj "$MMPROJ_PATH" \ -ngl 99 \ --host 0.0.0.0 \ --port 8000 \ --ctx-size 262144 \ --reasoning "on" \ --kv-unified \ -np 2 \ -fa "on" \ --no-mmap \ --image-min-tokens 1024 \ --spec-type "draft-mtp" \ --spec-draft-n-max 3 \ --spec-draft-p-min 0.75 \ -b 2048 \ -ub 512 ``` **Parameters:** - `--mmproj`: Multimodal projector for vision capabilities - `--reasoning "on"`: Enable reasoning mode - `--kv-unified`: Unified KV cache - `-np 2`: 2 parallel slots - `--spec-type "draft-mtp"`: Enable MTP speculative decoding - `--spec-draft-n-max 3`: Generate up to 3 dra...

Note30 octhuggingface.co/vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF

Source context: 482 downloads · 5 likes · Pipeline image-text-to-text · Repo vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF

Note14 agohuggingface.co/vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF

Source context: 703 downloads · 4 likes · Pipeline image-text-to-text · Repo vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF