Gemma 4 E4B It Heretic

--- library_name: transformers license: apache-2.0 license_link: https://ai.google.dev/gemma/docs/gemma_4_license pipeline_tag: any-to-any tags: - heretic - uncensored - decensored - abliterated --- <!-- aux-rescue-banner: begin --> > ## ⚠️ Update 2026-05-02 — please re-pull this model > > An earlier upload of this repo was missing `model.vision_tower.*`, `model.audio_tower.*` weights > (1160 tensors) due to a bug in `save_pretrained`: tensors not > registered in the loaded model class's `state_dict()` were silently > dropped on save. > > **Symptom:** Vision and audio tower weights were saved with flattened key paths and per-tensor quantization stats were dropped — both modalities loaded with random weights. Text-only inference was unaffected. > > **Status:** fixed on 2026-05-02 by uploading `model-auxiliary.safetensors` and > patching `model.safetensors.index.json`. The repo now carries an > `.aux_rescue.json` marker file so this fact is programmatically > verifiable. > > Source used for restoration: [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) > > If you cloned this repo before 2026-05-02, please pull again. > Repaired with [`aux-rescue`](https://github.com/timrohrbaugh/aux-rescue). <!-- aux-rescue-banner: end --> # This is a decensored version of [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0+custom ## Abliteration parameters | Parameter | Value | | :-------- | :---: | | **direction_index** | 16.63 | | **attn.o_proj.max_weight** | 0.88 | | **attn.o_proj.max_weight_position** | 34.05 | | **attn.o_proj.min_weight** | 0.72 | | **attn.o_proj.min_weight_distance** | 25.28 | | **mlp.down_proj.max_weight** | 1.94 | | **mlp.down_proj.max_weight_position** | 19.47 | | **mlp.down_proj.min_weight** | 1.32 | | **mlp.down_proj.min_weight_distance** | 27.32 | ## Performance | Metric | This model | Original model ([google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it)) | | :----- | :--------: | :---------------------------: | | **KL divergence** | 0.1063 | 0 *(by definition)* | | **Refusals** | 4/100 | 98/100 | ----- <div align="center"> <img src=https://ai.google.dev/gemma/images/gemma4_banner.png> </div> <p align="center"> <a href="https://huggingface.co/collections/google/gemma-4" target="_blank">Hugging Face</a> | <a href="...