Gemma 4 12B It Abliterated Uncensored

--- license: gemma license_link: https://ai.google.dev/gemma/docs/gemma_4_license library_name: transformers base_model: google/gemma-4-12B-it tags: - gemma - gemma4 - gemma-4 - unified - abliterated - uncensored - multimodal - vision - audio pipeline_tag: any-to-any --- <div align="center"> <img src="OYM_banner.png" alt="OpenYourMind" width="100%"> </div> # gemma-4-12B-it-abliterated-uncensored ## Overview Full BF16 weights of **gemma-4-12B-it-abliterated-uncensored** — an abliterated, uncensored variant of [google/gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it) (Gemma 4 12B Unified, dense, ~11.95B parameters). The model keeps Gemma 4's encoder-free **unified multimodal** stack intact — text, image, and audio inputs flow straight into a single decoder-only transformer — so this checkpoint is a drop-in replacement for the original instruction-tuned model at the architecture level. The pipeline: 1. **Refusal Ablation** — Residual-stream refusal directions (one per decoder layer) were extracted via diff-in-means on a labeled harmful/harmless prompt set and baked into the weights as a per-matrix delta on the residual-write modules, using our own custom abliteration framework. 2. **Multimodal Preservation** — Gemma 4 12B is encoder-free: image patches and audio waveforms are projected directly into the embedding space, so there is no separate vision/audio tower to graft back. Tensor names, shapes, and the `config.json` schema (`Gemma4UnifiedForConditionalGeneration`, `model_type: gemma4_unified`) match the base model exactly — this checkpoint loads anywhere the original loads. **Key Properties:** - Uncensored across the standard refusal axes - Reasoning preserved (configurable thinking mode — see Best Practices) - Multimodal: text + image + audio carried forward - Drop-in shape compatibility with `google/gemma-4-12B-it` ## Architecture | Property | Value | |----------|-------| | Architecture | `Gemma4UnifiedForConditionalGeneration` (`model_type: gemma4_unified`) | | Total Parameters | ~11.95B (dense) | | Decoder Layers | 48 | | Hidden Size | 3840 | | Attention | 16 heads / 8 KV heads, hybrid sliding-window (1024) + global (full) attention, p-RoPE | | Vocabulary | 262,144 | | Context Length | up to 256K tokens | | Modalities | Text, Image, Audio (encoder-free / unified) | ## Files | File | Description | Size | |------|------...