Huihui Qwen3 VL 4B Instruct Abliterated Comfy

Source / Provenance

• Base model: huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated (apache-2.0)
• Origin model: Qwen/Qwen3-VL-4B-Instruct
• Abliteration: only the text part was processed (not the vision part), so the visual encoder behaves identically to the upstream model.

What was done

BF16 single-file (*.safetensors, 8.88 GiB)

The original upstream repo ships the weights split across two safetensors shards (model-00001-of-00002.safetensors + model-00002-of-00002.safetensors). They were merged into a single safetensors file using the original model.safetensors.index.json mapping. No weights modified.

• 713 tensors
• dtype: bfloat16
• Verified structurally identical to upstream (same key set)

FP8 scaled (*-fp8_scaled.safetensors, 5.24 GiB)

Per-tensor abs-max quantization to float8_e4m3fn for the 252 linear projections of the language model (q/k/v/o_proj + gate/up/down_proj across all layers). Embeddings, layer norms, biases and the entire visual encoder stay in BF16.

• 1217 tensors (252 × 3 + 461 BF16)
• Quantised layers: float8_e4m3fn weights + float32 per-tensor scale + uint8[64] comfy_quant marker (JSON: {"format": "float8_e4m3fn", "full_precision_matrix_mult": false})
• Per-tensor scale = max(|w|) / 448 (E4M3FN max)
• Mean round-trip relative error ≈ 2.3% (typical for FP8 LLM quantisation)
• Schema matches the ComfyUI "fp8_scaled" convention used by other models in this size class (e.g. qwen3vl_4b_fp8_scaled.safetensors)

Quantisation script

The FP8 conversion was done on GPU (NVIDIA RTX 3090) in ~4 seconds. Script is available on request.

Usage

ComfyUI — Qwen3-VL Text Encoder (recommended)

1. Drop the *-fp8_scaled.safetensors into your ComfyUI models/text_encoders/ directory.
2. Use the Qwen3-VL Text Encoder node and select Huihui-Qwen3-VL-4B-Instruct-abliterated-fp8_scaled.
3. Pair with a Qwen3-VL compatible diffusion model and sampler.

ComfyUI — Full BF16 (when more precision is required)

1. Drop the *.safetensors into models/text_encoders/.
2. Use the same node but select the BF16 file. Higher VRAM usage (~16 GB on top of the diffusion model for FP16 diffusion).

transformers (BF16 only — tokenizer/configs are not bundled here)

The upstream repo huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated has the matching tokenizer, processor and configs. For BF16 inference:

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
import torch

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "ahmed22xa/Huihui-Qwen3-VL-4B-Instruct-abliterated-comfy",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated")

The FP8 file is not loadable with transformers.from_pretrained directly — it follows ComfyUI's per-tensor-FP8 layout with comfy_quant markers.

License & disclaimer

• License: apache-2.0 (inherited from upstream Qwen/Qwen3-VL-4B-Instruct).
• Abliteration notice: This is an uncensored variant. The safety filtering has been significantly reduced, potentially generating sensitive, controversial, or inappropriate content. Use with caution. See the upstream model card for the full disclaimer.
• No warranty. Users are solely responsible for any consequences arising from use of this model.