Qwen3.6 35B A3B Heretic GGUF

--- library_name: gguf license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE pipeline_tag: image-text-to-text tags: - heretic - uncensored - decensored - abliterated base_model: - llmfan46/Qwen3.6-35B-A3B-uncensored-heretic --- This repo contains the **EMB_BF16** quants, which are quantizations that preserve full precision for embeddings that always stay in system RAM. These embeddings are never copied to GPU VRAM, so they do not contribute to VRAM usage. They're lookup tables that are only accessed a few KB per token so they don't affect RAM bandwidth or computation efficiency either despite their larger size when unquantized. If you're not constrained by system RAM, having these embeddings in full precision may provide better accuracy. These embeddings are: - `token_embd.weight` - `position_embd.weight` - `token_types.weight` - `per_layer_token_embd.weight` For this model, `token_embd.weight` is left unquantized in the `EMB_BF16` quants. --- 🚨⚠️ I HAVE REACHED HUGGING FACE'S FREE STORAGE LIMIT ⚠️🚨 I can no longer upload new models unless I can cover the cost of additional storage. I host 70+ free models as an independent contributor and this work is unpaid. Without your support, no more new models can be uploaded. 🎉 Patreon (Monthly) | ☕ Ko-fi (One-time) Every contribution goes directly toward Hugging Face storage fees to keep models free for everyone. --- ### **88% fewer refusals** (10/100 Uncensored vs 83/100 Original) while preserving model quality (0.0015 KL divergence). ## ❤️ Support My Work Creating these models takes significant time, work and compute. If you find them useful consider supporting me:  | Platform | Link | What you get | |----------|------|--------------| | 🎉 Patreon | [Monthly support](https://patreon.com/LLMfan46) | Priority model requests | | ☕ Ko-fi | [One-time tip](https://ko-fi.com/llmfan46) | My eternal gratitude | Your help will motivate me and would go into further improving my workflow and coverings fees for storage, compute and may even help uncensoring bigger model with rental Cloud GPUs. ----- # This is a decensored version of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0 with a variant of the [Magnitude-Preserving Orthogonal Ablation (MPOA)](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration) method ## Abliteration parameters | Parameter | Value | | :-------- | :---: | | **direction_index** | 19.93 | | **attn.out_proj.max_weight** | 1.49 | | **attn.out_proj.max_weight_position** | 23.45 | | **attn.out_proj.min_weight** | 1.08 | | **attn.out_proj.min_weight_distance** | 16.54 | | **mlp.down_proj.max_weight** | 1.46 | | **mlp.down_proj.max_weight_position** | 28.05 | | **mlp.down_proj.min_weight** | 1.27 | | **mlp.down_proj.min_weight_distance** | 18.79 | | **attn.o_proj.max_weight** | 1.47 | | **attn.o_proj.max_weight_position** | 24.35 | | **attn.o_proj.min_weight** | 0.07 | | **attn.o_proj.min_weight_distance** | 22.58 | ## Targeted components * attn.o_proj * attn.out_proj * mlp.down_proj ## Performance | Metric | This model | Original model ([Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)) | | :----- | :--------: | :---------------------------: | | **KL divergence** | 0.0015 | 0 *(by definition)* | | **Refusals** | ✅ 10/100 | ❌ 83/100 | Lower refusals indicate fewer content restrictions, while lower KL divergence indicates more closeness to the original model's baseline. Higher refusals cause more rejections, objections, pushbacks, lecturing, censorship, softening and deflections. ## MMLU test results: Original: | Tasks |Version|Filter|n-shot|Metric| |Value | |Stderr| |---------------------------------------|------:|------|-----:|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.8372|± |0.0030| | - humanities | 2|none | |acc |↑ |0.7815|± |0.0058| | - formal_logic | 1|none | 0|acc |↑ |0.7381|± |0.0393| | - high_school_european_history | 1|none | 0|acc |↑ |0.8667|± |0.0265| | - high_school_us_history | 1|none | 0|acc |↑ |0.9265|± |0.0183| | - high_school_world_history | 1|none | 0|acc |↑ |0.9283|± |0.0168| | - international_law | 1|none | 0|acc |↑ |0.9339|± |0.0227| | - jurisprudence | 1|none | 0|acc |↑ |0.8889|± |0.0304| | - logical_fallacies | 1|none | 0|acc |↑ |0.9141|± |0.0220| | - moral_disputes | 1|none | 0|acc |↑ |0.8555|± |0.0189| | - moral_scenarios | 1|none | 0|acc |↑ |0.6425|± |0.0160| | - philosophy | 1|none | 0|acc |↑ |0.8810|± |0.0184| | - prehistory | 1|none | 0|acc |↑ |0.9043|± |0.0164| | - professional_law | 1|none | 0|acc |↑ |0.7047|± |0.0117| | - world_religions | 1|none | 0|acc |↑ |0.9064|± |0.0223| | - other | 2|none | |acc |↑ |0.8687|± |0.0057| | - business_ethics | 1|none | 0|acc |↑ |0.8900|± |0.0314| | - clinical_knowledge | 1|none | 0|acc |↑ |0.8981|± |0.0186| | - college_medicine | 1|none | 0|acc |↑ |0.8671|± |0.0259| | - global_facts | 1|none | 0|acc |↑ |0.5300|± |0.0502| | - human_aging | 1|none | 0|acc |↑ |0.8475|± |0.0241| | - management | 1|none | 0|acc |↑ |0.8835|± |0.0318| | - marketing | 1|none | 0|acc |↑ |0.9444|± |0.0150| | - medical_genetics | 1|none | 0|acc |↑ |0.9200|± |0.0273| | - miscellaneous | 1|none | 0|acc |↑ |0.9476|± |0.0080| | - nutrition | 1|none | 0|acc |↑ |0.8954|± |0.0175| | - professional_accounting | 1|none | 0|acc |↑ |0.7340|± |0.0264| | - professional_medicine | 1|none | 0|acc |↑ |0.9449|± |0.0139| | - virology | 1|none | 0|acc |↑ |0.5783|± |0.0384| | - social sciences | 2|none | |acc |↑ |0.9067|± |0.0052| | - econometrics | 1|none | 0|acc |↑ |0.7982|± |0.0378| | - high_school_geography | 1|none | 0|acc |↑ |0.9343|± |0.0176| | - high_school_government_and_politics| 1|none | 0|acc |↑ |0.9793|± |0.0103| | - high_school_macroeconomics | 1|none | 0|acc |↑ |0.8949|± |0.0156| | - high_school_microeconomics | 1|none | 0|acc |↑ |0.9538|± |0.0136| | - high_school_psychology | 1|none | 0|acc |↑ |0.9578|± |0.0086| | - human_sexuality | 1|none | 0|acc |↑ |0.8931|± |0.0271| | - professional_psychology | 1|none | 0|acc |↑ |0.8824|± |0.0130| | - public_relations | 1|none | 0|acc |↑ |0.7455|± |0.0417| | - security_studies | 1|none | 0|acc |↑ |0.8408|± |0.0234| | - sociology | 1|none | 0|acc |↑ |0.9303|± |0.0180| | - us_foreign_policy | 1|none | 0|acc |↑ |0.9500|± |0.0219| | - stem | 2|none | |acc |↑ |0.8214|± |0.0066| | - abstract_algebra | 1|none | 0|acc |↑ |0.7000|± |0.0461| | - anatomy | 1|none | 0|acc |↑ |0.8815|± |0.0279| | - astronomy | 1|none | 0|acc |↑ |0.9079|± |0.0235| | - college_biology | 1|none | 0|acc |↑ |0.9444|± |0.0192| | - college_chemistry | 1|none | 0|acc |↑ |0.6500|± |0.0479| | - college_computer_science | 1|none | 0|acc |↑ |0.7400|± |0.0441| | - college_mathematics | 1|none | 0|acc |↑ |0.6800|± |0.0469| | - college_physics | 1|none | 0|acc |↑ |0.6961|± |0.0458| | - computer_security | 1|none | 0|acc |↑ |0.8600|± |0.0349| | - conceptual_physics | 1|none | 0|acc |↑ |0.9489|± |0.0144| | - electrical_engineering | 1|none | 0|acc |↑ |0.8483|± |0.0299| | - elementary_mathematics | 1|none | 0|acc |↑ |0.8175|± |0.0199| | - high_school_biology | 1|none | 0|acc |↑ |0.9581|± |0.0114| | - high_school_chemistry | 1|none | 0|acc |↑ |0.8128|± |0.0274| | - high_school_computer_science | 1|none | 0|acc |↑ |0.9100|± |0.0288| | - high_school_mathematics | 1|none | 0|acc |↑ |0.6037|± |0.0298| | - high_school_physics | 1|none | 0|acc |↑ |0.8278|± |0.0308| | - high_school_statistics | 1|none | 0|acc |↑ |0.8194|± |0.0262| | - machine_learning | 1|none | 0|acc |↑ |0.8036|± |0.0377| | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |------------------|------:|------|------|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.8372|± |0.0030| | - humanities | 2|none | |acc |↑ |0.7815|± |0.0058| | - other | 2|none | |acc |↑ |0.8687|± |0.0057| | - social sciences| 2|none | |acc |↑ |0.9067|± |0.0052| | - stem | 2|none | |acc |↑ |0.8214|± |0.0066| Heretic: | Tasks |Version|Filter|n-shot|Metric| |Value | |Stderr| |---------------------------------------|------:|------|-----:|------|---|-----:|---|-----:| |mmlu | 2|none | |acc |↑ |0.8330|± |0.0030| | - humanities | 2|none | |acc |↑ |0.7692|± |0.0059| | - formal_logic | 1|none | 0|acc |↑ |0.6905|± |0.0413| | - high_school_european_history | 1|none | 0|acc |↑ |0.8606|± |0.0270| | - high_school_us_history | 1|none | 0|acc |↑ |0.9314|± |0.0177| | - high_school_world_history | 1|none | 0|acc |↑ |0.9156|± |0.0181| | - international_law | 1|none | 0|acc |↑ |0.9174|± |0.0251| | - jurisprudence | 1|none | 0|acc |↑ |0.8889|± |0.0304| | - logical_fallacies | 1|none | 0|acc |↑ |0.9264|± |0.0205| | - moral_disputes | 1|none | 0|acc |↑ |0.8439|± |0.0195| | - moral_scenarios | 1|none | 0|acc |↑ |0.6279|± |0.0162| | - philosophy | 1|none | 0|acc |↑ |0.8682|± |0.0192| | - prehistory | 1|none | 0|acc |↑ |0.9012|± |0.0166| | - professional_law | 1|none | 0|acc |↑ |0.6858|± |0.0119| | - world_religions | 1|none | 0|acc |↑ |0.9181|± |0.0210| | - other | 2|none | |acc |↑ |0.8680|± |0.0057| | - business_ethics | 1|none | 0|acc |↑ |0.8200|± |0.0386| | - clinical_knowledge | 1|none | 0|acc |↑ |0.8906|± |0.0192| | - college_medicine | 1|none | 0|acc |↑ |0.8844|± |0.0244| | - global_facts | 1|none | 0|acc |↑ |0.5500|± |0.0500| | - human_aging | 1|none | 0|acc |↑ |0.8296|± |0.0252| | - management | 1|none | 0|acc |↑ |0.9029|± |0.0293| | - marketing | 1|none | 0|acc |↑ |0.9444|± |0.0150| | - medical_genetics | 1|none | 0|acc |↑ |0.9300|± |0.0256| | - miscellaneous | 1|none | 0|acc |↑ |0.9553|± |0.0074| | - nutrition | 1|none | 0|acc |↑ |0.9020|± |0.0170| | - professional_accounting | 1|none | 0|acc |↑ |0.7270|± |0.0266| | - professional_medicine | 1|none | 0|acc |↑ |0.9375|± |0.0147| | - virology | 1|none | 0|acc |↑ |0.5723|± |0.0385| | - social sciences | 2|none | |acc |↑ |0.9071|± |0.0051| | - econometrics | 1|none | 0|acc |↑ |0.8070|± |0.0371| | - high_school_geography | 1|none | 0|acc |↑ |0.9444|± |0.0163| | - high_school_government_and_politics| 1|none | 0|acc |↑ |0.9793|± |0.0103| | - high_school_macroeconomics | 1|none | 0|acc |↑ |0.8923|± |0.0157| | - high_school_microeconomics | 1|none | 0|acc |↑ |0.9538|± |0.0136| | - high_school_psychology | 1|none | 0|acc |↑ |0.9560|± |0.0088| | - human_sexuality | 1|none | 0|acc |↑ |0.9084|± |0.0253| | - professional_psychology | 1|none | 0|acc |↑ |0.8807|± |0.0131| | - public_relations | 1|none | 0|acc |↑ |0.7364|± |0.0422| | - security_studies | 1|none | 0|acc |↑ |0.8408|± |0.0234| | - sociology | 1|none | 0|acc |↑ |0.9303|± |0.0180| | - us_foreign_policy | 1|none | 0|acc |↑ |0.9500|± |0.0219| | - stem | 2|none | |acc |↑ |0.8214|± |0.0066| | - abstract_algebra | 1|none | 0|acc |↑ |0.6500|± |0.0479| | - anatomy | 1|none | 0|acc |↑ |0.8741|± |0.0287| | - astronomy | 1|none | 0|acc |↑ |0.9145|± |0.0228| | - college_biology | 1|none | 0|acc |↑ |0.9444|± |0.0192| | - college_chemistry | 1|none | 0|acc |↑ |0.6600|± |0.0476| | - college_computer_science | 1|none | 0|acc |↑ |0.7400|± |0.0441| | - college_mathematics | 1|none | 0|acc |↑ |0.6500|± |0.0479| | - college_physics | 1|none | 0|acc |↑ |0.6765|± |0.0466| | - computer_security | 1|none | 0|acc |↑ |0.8400|± |0.0368| | - conceptual_physics | 1|none | 0|acc |↑ |0.9447|± |0.0149| | - electrical_engineering | 1|none | 0|acc |↑ |0.8483|± |0.0299| | - elementary_mathematics | 1|none | 0|acc |↑ |0.8307|± |0.0193| | - high_school_biology | 1|none | 0|acc |↑ |0.9645|± |0.0105| | - high_school_chemistry | 1|none | 0|acc |↑ |0.8079|± |0.0277| | - high_school_computer_science | 1|none | 0|acc |↑ |0.9300|± |0.0256| | - high_school_mathematics | 1|none | 0|acc |↑ |0.6222|± |0.0296| | - high_school_physics | 1|none | 0|acc |↑ |0.8278|± |0.0308| | - high_school_statistics | 1|none | 0|acc |↑ |0.8148|± |0.0265| | -...