ModelÜberprüftExtern

Qwen3.6 27B AEON Ultimate Uncensored Q6K MTP GGUF

GGUF Q6K quantization of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 with MTP (Multi-Token Prediction) weights included.

Quellehuggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF/tree/c9e70460830b09512cf264c2c58994d812d9c216

ArtOtherBasismodellAEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16VersionRevision c9e7046Verleger@pyysCBewertung

Modellquelle

Art: Other
Dateigröße: 20,9 GB
Basismodell: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
Version: vRevision c9e7046
Aktualisiert: 3. Okt. 2026
Verleger: @pyys
Downloads: 102
Parameter: 27B
Quelle: Hugging Face

Quellenbeschreibung

Qwen3.6-27B-AEON-Ultimate-Uncensored — Q6_K GGUF + MTP

GGUF Q6_K quantization of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 with MTP (Multi-Token Prediction) weights included.

Key Features

Q6_K quantization — near-lossless quality, practical for consumer hardware
MTP weights included (866 tensors) — enables speculative decoding via --spec-type draft-mtp in llama.cpp for significantly faster inference
Vision supported — mmproj file (931MB) available in separate repo for multimodal image input

Model Details

Property	Value
Base Model	AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
Architecture	Qwen 3.6 (Hybrid Linear Attention + Full Attention, 64 layers)
Parameters	27B
Quantization	Q6_K
Main Model Size	21.4 GB
mmproj Size	931 MB
Main Model Tensors	866 (including MTP extension)
mmproj Tensors	334 (27 ViT blocks + merger + patch embed + position embed)
Context Length	262,144 tokens (training)
License	Apache 2.0

Quantization Process

BF16 original → FP16 GGUF conversion (V100 does not support BF16)
FP16 GGUF → Q6_K quantization with MTP weights preserved

How to Use

llama.cpp (recommended)

# Text-only
llama-server \
  -m qwen3.6-27b-Q6K-mtp.gguf \
  -ngl 99 \
  -c 80000 \
  --host 0.0.0.0 \
  --port 8080 \
  --spec-type draft-mtp

# With vision (download mmproj from https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF)
llama-server \
  -m qwen3.6-27b-Q6K-mtp.gguf \
  --mmproj qwen3.6-27b-mmproj.gguf \
  -ngl 99 \
  -c 80000 \
  --host 0.0.0.0 \
  --port 8080 \
  --spec-type draft-mtp

MTP (Multi-Token Prediction)

MTP enables speculative decoding without a separate draft model. The MTP weights are baked into this GGUF file. Use --spec-type draft-mtp to activate.

Performance

Tested on V100 SXM2 32GB:

Configuration	TPS	Notes
Q6_K + MTP	~31.5 t/s	MTP acceptance rate ~41%
Q6_K + MTP (via OpenAI-compatible API)	~39.5 t/s	Acceptance rate ~61%

Files

File	Size	Description
`qwen3.6-27b-Q6K-mtp.gguf`	21.4 GB	Main model (Q6_K + MTP weights)
`qwen3.6-27b-mmproj.gguf`	931 MB	Vision projector (separate repo)

Notes

V100 GPUs do not support BF16 — this model was converted via FP16 intermediate
For single-GPU deployment, use --split-mode none -mg 0 to keep all weights on one GPU
KV cache of 80K tokens fits within a single V100 32GB alongside the model

Credits

Original model: AEON-7 — Qwen3.6-27B-AEON-Ultimate-Uncensored
Base architecture: Qwen — Qwen3.6-27B
Quantization tooling: llama.cpp

Quellen

1 Quelle

Hugging FacePrimärVersion c9e70460830b09512cf264c2c58994d812d9c216

huggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF/tree/c9e70460830b09512cf264c2c58994d812d9c216

Verifiziert 3. Okt.

Modellartefakte

1 Artefakt

qwen3.6-27b-Q6K-mtp.gguf

gguf · 20,9 GB · SHA-256 0f354dcd6839…c96b · Hugging Face

Herunterladen

Quellenauszüge

3 Auszüge

Readme11. Julihuggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

--- license: apache-2.0 base_model: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 tags: - gguf - qwen3 - uncensored - mtp - llama-cpp - quantized - multimodal - vision model_type: qwen3 quantized_by: lllabs model_size: 27B num_params: 27320697856 --- # Qwen3.6-27B-AEON-Ultimate-Uncensored — Q6_K GGUF + MTP GGUF Q6_K quantization of [AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16](https://huggingface.co/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16) with **MTP (Multi-Token Prediction) weights included**. ## Key Features - **Q6_K quantization** — near-lossless quality, practical for consumer hardware - **MTP weights included** (866 tensors) — enables speculative decoding via `--spec-type draft-mtp` in llama.cpp for significantly faster inference - **Vision supported** — mmproj file (931MB) available in [separate repo](https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF) for multimodal image input ## Model Details | Property | Value | |---|---| | Base Model | AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 | | Architecture | Qwen 3.6 (Hybrid Linear Attention + Full Attention, 64 layers) | | Parameters | 27B | | Quantization | Q6_K | | Main Model Size | 21.4 GB | | mmproj Size | 931 MB | | Main Model Tensors | 866 (including MTP extension) | | mmproj Tensors | 334 (27 ViT blocks + merger + patch embed + position embed) | | Context Length | 262,144 tokens (training) | | License | Apache 2.0 | ## Quantization Process 1. BF16 original → FP16 GGUF conversion (V100 does not support BF16) 2. FP16 GGUF → Q6_K quantization with MTP weights preserved ## How to Use ### llama.cpp (recommended) ```bash # Text-only llama-server \ -m qwen3.6-27b-Q6K-mtp.gguf \ -ngl 99 \ -c 80000 \ --host 0.0.0.0 \ --port 8080 \ --spec-type draft-mtp # With vision (download mmproj from https://huggingface.co/pyys/Qwen3.6-27B-mmproj-GGUF) llama-server \ -m qwen3.6-27b-Q6K-mtp.gguf \ --mmproj qwen3.6-27b-mmproj.gguf \ -ngl 99 \ -c 80000 \ --host 0.0.0.0 \ --port 8080 \ --spec-type draft-mtp ``` ### MTP (Multi-Token Prediction) MTP enables speculative decoding without a separate draft model. The MTP weights are baked into this GGUF file. Use `--spec-type draft-mtp` to activate. ## Performance Tested on V100 SXM2 32GB: | Configuration | TPS | Notes | |---|---|---| | Q6_K + MTP | ~31.5 t/s | MTP acceptance rate ~41% | | Q6_K + MTP (via OpenAI-compat...

Note3. Okt.huggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

Source context: 102 downloads · 1 likes · Repo pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

Note17. Julihuggingface.co/pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF

Source context: 614 downloads · 0 likes · Repo pyys/Qwen3.6-27B-AEON-Ultimate-Uncensored-Q6K-MTP-GGUF