Qwen3.6 27B AEON Ultimate Uncensored NVFP4

[!IMPORTANT] Superseded by Qwen3.8 — use this instead: AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED

Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX): AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED

This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports.

Serve with the unified container: ghcr.io/aeon-7/aeon-vllm-ultimate:latest (Spark / GB10) or ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest (RTX discrete).

Qwen3.6-27B-AEON-Ultimate-Uncensored-NVFP4

AEON Qwen — Supreme Being of the Digital Cosmos

NVFP4 hardware-quantized release of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored. Same model, same 0/50 refusal rate, same preserved capabilities — now compressed from 51 GB BF16 to 26 GB NVFP4 for native FP4 hardware throughput on DGX Spark / GB10 / Blackwell sm_121a.

This release is multimodal-preserved (vision tower stays BF16 — text + image inference fully functional) and hybrid-attention-preserved (the 48 linear-attention / GatedDeltaNet layers stay BF16; FP4 only applies to the 16 full-attention layers' output projections + all MLPs).

---

What Changed vs BF16

Aspect	BF16 (source)	NVFP4 (this release)
Disk size	51 GB	26 GB (49% reduction)
Refusal rate	0/50	inherited — to be verified post-deploy
Multimodal	preserved	preserved (vision BF16, no degradation)
Hybrid SSM	repaired + intact	intact (linear_attn BF16-preserved)
Hardware target	A100 / H100 / RTX PRO 6000 BF16	DGX Spark (GB10), B100/B200, RTX PRO 6000 Blackwell with native FP4 throughput
KL vs BF16 source	n/a	expected ≤0.001 (typical for this recipe class)

The NVFP4 quantization scheme is NVIDIA-mandated: E2M1 element format, block_size=16, FP8 E4M3 per-block scales, FP32 per-tensor scale, symmetric signed.

---

Quantization Recipe

Tool: llm-compressor 0.10.1.dev107 (vllm-project) using QuantizationModifier(scheme="NVFP4") post-training quantization.

from llmcompressor.modifiers.quantization import QuantizationModifier
recipe = QuantizationModifier(
    targets="Linear",
    scheme="NVFP4",
    ignore=[
        "lm_head",                  # always
        "re:.*embed_tokens.*",      # always
        "re:.*\\.visual\\..*",      # vision tower BF16 — preserves multimodal
        "re:.*visual\\..*",
        "re:.*linear_attn\\..*",    # SSM/GDN BF16 — Mamba state collapses under FP4
        "re:.*norm.*",
        "re:.*q_norm.*",
        "re:.*k_norm.*",
    ],
)

Calibration: open-platypus, 512 samples × 4096 tokens. Pipeline: sequential with sequential_targets=["Qwen3_5DecoderLayer"] — required for hybrid stacks (mixed full + linear attention layers); without explicit targeting, llm-compressor's auto-discovery silently skips layers. Loader: AutoModelForImageTextToText to preserve the Qwen3_5ForConditionalGeneration multimodal class. Processor: passed explicitly to oneshot() to avoid the "model processor required when a dataset is provided" failure on multimodal builds without torchvision.

Verification (pass):

• 1 shard, 1952 keys
• 64 quantized full-attention projections (16 layers × 4 q/k/v/o)
• 432 linear_attn.* keys preserved BF16 (48 layers × 9 modules)
• 333 visual.* keys preserved BF16 (vision tower intact)
• 319 norm keys preserved BF16
• lm_head and embed_tokens preserved BF16
• NVFP4-packed weights present
• input_global_scale magnitudes 142–346 (healthy range)

Wall-clock quant time: ~57 minutes on 1× RTX PRO 6000 Blackwell (96 GB).

---

Deployment

vLLM on DGX Spark (GB10 / sm_121a) — recommended

Serve on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container (= tag :2026-06-18-v0.23.0-dflashfix; rollback :2026-06-11-pr41703) with the external z-lab DFlash drafter at the validated default num_speculative_tokens: 10. The patched CUTLASS NVFP4 path uses native FP4 tensor-core kernels and outperforms the Marlin fallback — do NOT force VLLM_NVFP4_GEMM_BACKEND=marlin (that's the workaround for stock vLLM builds where CUTLASS is broken on SM121).

docker pull ghcr.io/aeon-7/aeon-vllm-ultimate:latest
hf download z-lab/Qwen3.6-27B-DFlash --local-dir ./dflash-drafter

# ENTRYPOINT is /bin/bash → pass --entrypoint vllm then serve ...
docker run --gpus all --ipc=host --network=host \
  -e TORCH_CUDA_ARCH_LIST=12.1a -e ENABLE_NVFP4_SM100=0 \
  -e VLLM_USE_FLASHINFER_MOE_FP4=0 -e VLLM_USE_FLASHINFER_SAMPLER=1 \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  -v /path/to/model:/models/aeon-ultimate:ro \
  -v ./dflash-drafter:/models/dflash-drafter:ro \
  --entrypoint vllm ghcr.io/aeon-7/aeon-vllm-ultimate:latest \
  serve /models/aeon-ultimate \
    --served-model-name aeon-ultimate qwen36-ultimate aeon-fast aeon-deep \
    --host 0.0.0.0 --port 8000 \
    --quantization compressed-tensors \
    --mamba-cache-dtype float32 \
    --max-model-len 256000 \
    --max-num-seqs 64 \
    --max-num-batched-tokens 16384 \
    --gpu-memory-utilization 0.65 \
    --enable-chunked-prefill \
    --enable-prefix-caching \
    --load-format safetensors \
    --trust-remote-code \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --reasoning-parser qwen3 \
    --attention-backend flash_attn \
    --limit-mm-per-prompt '{"image":4,"video":2}' \
    --mm-encoder-tp-mode data \
    --mm-processor-cache-type shm \
    --speculative-config '{"method":"dflash","model":"/models/dflash-drafter","num_speculative_tokens":10}'

--gpu-memory-utilization is 0.65 solo with DFlash / 0.6 when ASR/TTS/embeddings share the Spark; on GB10 unified memory keep it in the 0.6–0.7 band — above ~0.8 the shared CPU+GPU pool page-thrashes and stalls the box (even 0.85 stalls), and DFlash's verify buffers aren't counted by the fraction, so leave headroom. Use --mamba-cache-dtype float32 (more precise recurrent state + slightly higher DFlash acceptance than float16) and omit --mamba-block-size (the default lowers single-stream TTFT vs 256). Do not set --kv-cache-dtype with DFlash — the non-causal drafter requires BF16 KV. For the XS body (21 GB, tighter cards), see the -Multimodal-NVFP4-MTP-XS sibling; full config reference is in the DFlash repo.

Four served-model-name aliases — same weights, different sampling conventions

vLLM's --served-model-name accepts multiple values; each becomes a separate model ID in /v1/models but all four route to the same /models/aeon-ultimate backend. The differentiation lives in client-side sampling conventions, exploiting a hard property of DFlash speculative decoding:

DFlash's drafter is trained to match the target's argmax. Greedy (T=0) sampling lets the drafter hit ~80 % first-position acceptance → ~3× speedup. Sampled (T≥0.7) sampling drops acceptance to ~5 % → speedup collapses. Splitting the model into two API IDs lets agent runtimes route per-workload without changing endpoints mid-conversation.

served name	Recommended sampling	Use case	DFlash effect (single-stream)
aeon-fast	temperature=0, top_p=1.0 (greedy)	tool calls, agent loops, code, math, structured / JSON output	~80 % drafter acceptance → ~91 tok/s (Spark v2 measured)
aeon-deep	temperature=0.7, top_p=0.95, top_k=64, repetition_penalty=1.05	creative writing, brainstorming, open-ended Q&A, character roleplay	~5 % acceptance → ~38 tok/s; variety prioritized over throughput
aeon-ultimate	client-provided (no opinion injected)	raw-API consumers that bring their own sampling	depends on client params
qwen36-ultimate	client-provided (no opinion injected)	same as aeon-ultimate, vendor-agnostic Qwen-branded alias for tools that key off the qwen prefix	depends on client params

All four aliases share the same 256 K context window, full multimodal pipeline (image × 4, video × 2), --reasoning-parser qwen3 thinking-mode support, and --tool-call-parser qwen3_coder tool-call output. Only the name and the sampling defaults the client attaches differ.

Quick routing reference (agent runtimes)

# Pseudocode for the routing decision
def pick_model(workload):
    if workload in {"tool_call", "code_gen", "math", "json", "structured"}:
        return "aeon-fast"        # T=0, max DFlash speedup
    elif workload in {"creative", "brainstorm", "open_qa", "roleplay"}:
        return "aeon-deep"        # T=0.7, variety
    else:
        return "aeon-ultimate"    # client controls sampling

For a worked OpenClaw integration that registers both aeon-fast and aeon-deep as separate provider entries with these defaults, see docs/openclaw.md in the deployment repo.

Python (transformers) — for testing or non-vLLM serving

from transformers import AutoModelForImageTextToText, AutoTokenizer
import torch

model_id = "AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-NVFP4"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,   # vision tower + non-quantized weights
    device_map="cuda:0",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "Your prompt here"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Requires compressed-tensors >= 0.12 for NVFP4 dequant on the fly.

---

Hardware notes

Hardware	Notes
DGX Spark (GB10, sm_121a)	Primary target. Use patched vLLM CUTLASS path. Expect ~50 tok/s single-stream after warmup.
B100 / B200 (sm_100)	Native FP4 compute via tcgen05/UTCQMMA — fastest hardware for this format.
RTX PRO 6000 Blackwell (sm_120)	Native FP4 via CUTLASS path. Excellent throughput.
A100 / H100 (sm_80, sm_90)	NVFP4 dequantizes to BF16/FP8 at kernel level — works but no FP4 throughput advantage. Use BF16 release instead for best perf on these.

---

Provenance

• BF16 source: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored — see source card for full pipeline (FernflowerAI SSM repair → abliterix-v1.4 abliteration → trial 46 of 50 selected for capability preservation).
• Original base: Qwen/Qwen3.6-27B by Alibaba.
• Quantization tool: llm-compressor by...