Aller au contenu
ModelVérifiéExterne
Télécharger

Qwen3.6 27B AEON Ultimate Uncensored Q6K MTP GGUF

GGUF Q6K quantization of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 with MTP (Multi-Token Prediction) weights included.

Sourcehuggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF/tree/c9e70460830b09512cf264c2c58994d812d9c216
TypeOtherModèle de baseAEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16VersionRevision c9e7046Éditeur@pyysCClasse

Source du modèle

Type
Other
Taille du fichier
20,9 GB
Modèle de base
AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
Version
vRevision c9e7046
Mis à jour
3 oct. 2026
Éditeur
@pyys
Téléchargements
102
Paramètres
27B
Source
Hugging Face

Description de la source

Qwen3.6-27B-AEON-Ultimate-Uncensored — Q6_K GGUF + MTP

GGUF Q6_K quantization of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 with MTP (Multi-Token Prediction) weights included.

Key Features

  • Q6_K quantization — near-lossless quality, practical for consumer hardware
  • MTP weights included (866 tensors) — enables speculative decoding via --spec-type draft-mtp in llama.cpp for significantly faster inference
  • Vision supported — mmproj file (931MB) available in separate repo for multimodal image input

Model Details

PropertyValue
Base ModelAEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
ArchitectureQwen 3.6 (Hybrid Linear Attention + Full Attention, 64 layers)
Parameters27B
QuantizationQ6_K
Main Model Size21.4 GB
mmproj Size931 MB
Main Model Tensors866 (including MTP extension)
mmproj Tensors334 (27 ViT blocks + merger + patch embed + position embed)
Context Length262,144 tokens (training)
LicenseApache 2.0

Quantization Process

  1. BF16 original → FP16 GGUF conversion (V100 does not support BF16)
  2. FP16 GGUF → Q6_K quantization with MTP weights preserved

How to Use

llama.cpp (recommended)

# Text-only
llama-server \
  -m qwen3.6-27b-Q6K-mtp.gguf \
  -ngl 99 \
  -c 80000 \
  --host 0.0.0.0 \
  --port 8080 \
  --spec-type draft-mtp

# With vision (download mmproj from https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF)
llama-server \
  -m qwen3.6-27b-Q6K-mtp.gguf \
  --mmproj qwen3.6-27b-mmproj.gguf \
  -ngl 99 \
  -c 80000 \
  --host 0.0.0.0 \
  --port 8080 \
  --spec-type draft-mtp

MTP (Multi-Token Prediction)

MTP enables speculative decoding without a separate draft model. The MTP weights are baked into this GGUF file. Use --spec-type draft-mtp to activate.

Performance

Tested on V100 SXM2 32GB:

ConfigurationTPSNotes
Q6_K + MTP~31.5 t/sMTP acceptance rate ~41%
Q6_K + MTP (via OpenAI-compatible API)~39.5 t/sAcceptance rate ~61%

Files

FileSizeDescription
qwen3.6-27b-Q6K-mtp.gguf21.4 GBMain model (Q6_K + MTP weights)
qwen3.6-27b-mmproj.gguf931 MBVision projector (separate repo)

Notes

  • V100 GPUs do not support BF16 — this model was converted via FP16 intermediate
  • For single-GPU deployment, use --split-mode none -mg 0 to keep all weights on one GPU
  • KV cache of 80K tokens fits within a single V100 32GB alongside the model

Credits

  • Original model: AEON-7 — Qwen3.6-27B-AEON-Ultimate-Uncensored
  • Base architecture: Qwen — Qwen3.6-27B
  • Quantization tooling: llama.cpp

Sources

1 source
Hugging FacePrincipalVersion c9e70460830b09512cf264c2c58994d812d9c216
huggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF/tree/c9e70460830b09512cf264c2c58994d812d9c216

Vérifié 3 oct.

Artefacts du modèle

1 artefact

qwen3.6-27b-Q6K-mtp.gguf

gguf · 20,9 GB · SHA-256 0f354dcd6839…c96b · Hugging Face

Télécharger

Extraits de sources

3 extraits
Readme11 juil.huggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

--- license: apache-2.0 base_model: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 tags: - gguf - qwen3 - uncensored - mtp - llama-cpp - quantized - multimodal - vision model_type: qwen3 quantized_by: lllabs model_size: 27B num_params: 27320697856 --- # Qwen3.6-27B-AEON-Ultimate-Uncensored — Q6_K GGUF + MTP GGUF Q6_K quantization of [AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16](https://huggingface.co/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16) with **MTP (Multi-Token Prediction) weights included**. ## Key Features - **Q6_K quantization** — near-lossless quality, practical for consumer hardware - **MTP weights included** (866 tensors) — enables speculative decoding via `--spec-type draft-mtp` in llama.cpp for significantly faster inference - **Vision supported** — mmproj file (931MB) available in [separate repo](https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF) for multimodal image input ## Model Details | Property | Value | |---|---| | Base Model | AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 | | Architecture | Qwen 3.6 (Hybrid Linear Attention + Full Attention, 64 layers) | | Parameters | 27B | | Quantization | Q6_K | | Main Model Size | 21.4 GB | | mmproj Size | 931 MB | | Main Model Tensors | 866 (including MTP extension) | | mmproj Tensors | 334 (27 ViT blocks + merger + patch embed + position embed) | | Context Length | 262,144 tokens (training) | | License | Apache 2.0 | ## Quantization Process 1. BF16 original → FP16 GGUF conversion (V100 does not support BF16) 2. FP16 GGUF → Q6_K quantization with MTP weights preserved ## How to Use ### llama.cpp (recommended) ```bash # Text-only llama-server \ -m qwen3.6-27b-Q6K-mtp.gguf \ -ngl 99 \ -c 80000 \ --host 0.0.0.0 \ --port 8080 \ --spec-type draft-mtp # With vision (download mmproj from https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF) llama-server \ -m qwen3.6-27b-Q6K-mtp.gguf \ --mmproj qwen3.6-27b-mmproj.gguf \ -ngl 99 \ -c 80000 \ --host 0.0.0.0 \ --port 8080 \ --spec-type draft-mtp ``` ### MTP (Multi-Token Prediction) MTP enables speculative decoding without a separate draft model. The MTP weights are baked into this GGUF file. Use `--spec-type draft-mtp` to activate. ## Performance Tested on V100 SXM2 32GB: | Configuration | TPS | Notes | |---|---|---| | Q6_K + MTP | ~31.5 t/s | MTP acceptance rate ~41% | | Q6_K + MTP (via OpenAI-compat...

Note3 oct.huggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

Source context: 102 downloads · 1 likes · Repo pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

Note17 juil.huggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

Source context: 614 downloads · 0 likes · Repo pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF