ThinkingCap Qwen3.6 27B IQ4 XS Pure GGUF

ThinkingCap Qwen3.6-27B Quantized to IQ4_XS for 16 GB VRAM GPUs

This GGUF is generated with identical approach taken by Ununnilium's Qwen3.6-27B-IQ4_XS-pure-GGUF but targetting bottlecapai's ThinkingCap variant of Qwen3.6.

llama-quantize --pure \
    ThinkingCap-Qwen3.6-27B-f16.gguf \
    thinkingcap-qwen3.6-27b-IQ4_XS-pure.gguf \
    IQ4_XS \
    $(nproc)

For additional details see bottlecapai's ThinkingCap article. For recommended Qwen3.6 parameters and deployment see the Unsloth Qwen3.6 Guide.

---

bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF

GGUF / llama.cpp quantizations of bottlecapai/ThinkingCap-Qwen3.6-27B — capability of Qwen3.6-27B with 50% less thinking tokens on average, achieved by finetuning Qwen3.6-27B (Qwen Team, 2026) with online reinforcement learning while preserving the original answer quality and style.

➡️ Full model description, evaluation results (multi-seed, statistically tested), recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.6-27B.

About GGUF and quantization

GGUF is a single-file model format for running LLMs locally with llama.cpp and compatible runtimes (Ollama, LM Studio, ...). The quantized variants below store weights at reduced precision — e.g. ~4.7 bits per weight for Q4_K_M instead of the 16-bit f16 source — cutting download size and memory severalfold at a small, measured quality cost.

Files

File	Quant	Size
ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf	Q4_K_M	15.7 GB
ThinkingCap-Qwen3.6-27B-Q8_0.gguf	Q8_0	27.1 GB
ThinkingCap-Qwen3.6-27B-f16.gguf	f16	50.9 GB
mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf	mmproj (vision)	0.9 GB

f16 is the unquantized source; Q8_0 is near-lossless; Q4_K_M is the recommended size/quality balance for most local setups.

When shown, Mean KLD is the mean KL divergence of the quant vs the f16 model over a held-out corpus (lower = closer to full precision).

Usage (llama.cpp)

# pull a specific quant straight from the Hub and chat
llama-cli -hf bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF:Q4_K_M -p "Hi"

# or download one file and run it
huggingface-cli download bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf --local-dir .
llama-cli -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf -p "Hi"

Vision (image input)

ThinkingCap is a vision-language model. Image input needs the multimodal projector mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf (in this repo) loaded alongside a text GGUF — the single f16 mmproj pairs with any of the quants above.

• LM Studio / Jan / Ollama, ...: download the mmproj-*.gguf from this repo; LM Studio auto-detects it and enables the image (🖼️) button.
• llama.cpp CLI:

huggingface-cli download bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF \
  ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf --local-dir .
llama-mtmd-cli -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf \
  --mmproj mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf --image photo.jpg -p "Describe this image."

• llama-server: add --mmproj mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf to expose an OpenAI-compatible vision endpoint.