Zum Inhalt springen
ModelÜberprüftExtern
Herunterladen

Gemma 4 12b It Apostate I1 GGUF

For a convenient overview and download list, visit our model page for this model.

Quellehuggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/tree/5062194730aa822f66521301aead4c77038c34eb
ArtOtherBasismodellheterodoxin/gemma-4-12b-it-apostateVersionRevision 5062194Verleger@mradermacherCBewertung

Modellquelle

Art
Other
Dateigröße
7,13 MB–9,88 GB
Basismodell
heterodoxin/gemma-4-12b-it-apostate
Version
vRevision 5062194
Aktualisiert
3. Okt. 2026
Verleger
@mradermacher
Downloads
1458
Parameter
12B
Quelle
Hugging Face

Quellenbeschreibung

About

weighted/imatrix quants of https://huggingface.co/heterodoxin/gemma-4-12b-it-apostate

For a convenient overview and download list, visit our model page for this model.

static quants are available at https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-GGUF

This is a vision model - mmproj files (if any) will be in the static repository.

Usage

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

LinkTypeSize/GBNotes
GGUFimatrix0.1imatrix file (for creating your own quants)
GGUFi1-IQ1_S3.4for the desperate
GGUFi1-IQ1_M3.6mostly desperate
GGUFi1-IQ2_XXS4.0
GGUFi1-IQ2_XS4.3
GGUFi1-IQ2_S4.6
GGUFi1-IQ2_M4.9
GGUFi1-Q2_K_S4.9very low quality
GGUFi1-Q2_K5.3IQ3_XXS probably better
GGUFi1-IQ3_XXS5.4lower quality
GGUFi1-IQ3_XS5.8
GGUFi1-IQ3_S6.1beats Q3_K*
GGUFi1-Q3_K_S6.1IQ3_XS probably better
GGUFi1-IQ3_M6.3
GGUFi1-Q3_K_M6.6IQ3_S probably better
GGUFi1-Q3_K_L7.1IQ3_M probably better
GGUFi1-IQ4_XS7.3
GGUFi1-IQ4_NL7.6prefer IQ4_XS
GGUFi1-Q4_07.7fast, low quality
GGUFi1-Q4_K_S7.7optimal size/speed/quality
GGUFi1-Q4_K_M8.0fast, recommended
GGUFi1-Q4_18.4
GGUFi1-Q5_K_S9.1
GGUFi1-Q5_K_M9.3
GGUFi1-Q6_K10.7practically like static Q6_K

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

image.png

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

Quellen

1 Quelle
Hugging FacePrimärVersion 5062194730aa822f66521301aead4c77038c34eb
huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/tree/5062194730aa822f66521301aead4c77038c34eb

Verifiziert 3. Okt.

Modellartefakte

25 Artefakte

gemma-4-12b-it-apostate.i1-IQ1_M.gguf

gguf · 3,29 GB · SHA-256 aca5519e81b6…d7cd · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ1_S.gguf

gguf · 3,09 GB · SHA-256 5521d0da8f4d…2960 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ2_M.gguf

gguf · 4,47 GB · SHA-256 23cbf026162a…80d7 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ2_S.gguf

gguf · 4,20 GB · SHA-256 f5ec45db8542…b48c · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ2_XS.gguf

gguf · 3,93 GB · SHA-256 3c04d0243f5d…5991 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ2_XXS.gguf

gguf · 3,63 GB · SHA-256 2841dbb355c8…70cf · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ3_M.gguf

gguf · 5,74 GB · SHA-256 a5deb1dac0fc…95c8 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ3_S.gguf

gguf · 5,55 GB · SHA-256 e6b54197b7ce…c437 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ3_XS.gguf

gguf · 5,31 GB · SHA-256 4f233aba827b…42fc · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ3_XXS.gguf

gguf · 4,92 GB · SHA-256 052098fa5146…4235 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ4_NL.gguf

gguf · 7,02 GB · SHA-256 c26a0be2f590…75b9 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-IQ4_XS.gguf

gguf · 6,68 GB · SHA-256 d332a2328ea5…83ec · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q2_K_S.gguf

gguf · 4,50 GB · SHA-256 c834539616ef…6e99 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q2_K.gguf

gguf · 4,81 GB · SHA-256 40a4b0bafe3f…5558 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q3_K_L.gguf

gguf · 6,52 GB · SHA-256 58b1c074bd5d…6966 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q3_K_M.gguf

gguf · 6,07 GB · SHA-256 256b2e05b9ce…d389 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q3_K_S.gguf

gguf · 5,55 GB · SHA-256 9a4654cb6928…7c99 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q4_0.gguf

gguf · 7,04 GB · SHA-256 16db47d881f1…2cbb · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q4_1.gguf

gguf · 7,72 GB · SHA-256 03f11390342e…0571 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q4_K_M.gguf

gguf · 7,40 GB · SHA-256 c41471e2acfc…4aa6 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q4_K_S.gguf

gguf · 7,07 GB · SHA-256 0a321dca9a18…b20f · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q5_K_M.gguf

gguf · 8,60 GB · SHA-256 23db7894d2cf…9cc9 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q5_K_S.gguf

gguf · 8,41 GB · SHA-256 fcabe4d6ad72…3aa6 · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.i1-Q6_K.gguf

gguf · 9,88 GB · SHA-256 ca6eb393e0d3…9f2f · Hugging Face

Herunterladen

gemma-4-12b-it-apostate.imatrix.gguf

gguf · 7,13 MB · SHA-256 92ca89d25769…1e78 · Hugging Face

Herunterladen

Quellenauszüge

3 Auszüge
Readme11. Julihuggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF

--- base_model: heterodoxin/gemma-4-12b-it-apostate language: - en library_name: transformers mradermacher: readme_rev: 1 quantized_by: mradermacher tags: - apostate - uncensored - abliteration --- ## About <!-- ### quantize_version: 2 --> <!-- ### output_tensor_quantised: 1 --> <!-- ### convert_type: hf --> <!-- ### vocab_type: --> <!-- ### tags: nicoboss --> <!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S --> <!-- ### quants_skip: --> <!-- ### skip_mmproj: 1 --> weighted/imatrix quants of https://huggingface.co/heterodoxin/gemma-4-12b-it-apostate <!-- provided-files --> ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#gemma-4-12b-it-apostate-i1-GGUF).*** static quants are available at https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-GGUF **This is a vision model - mmproj files (if any) will be in the [static repository](https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-GGUF).** ## Usage If you are unsure how to use GGUF files, refer to one of [TheBloke's READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for more details, including on how to concatenate multi-part files. ## Provided Quants (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | [GGUF](https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/resolve/main/gemma-4-12b-it-apostate.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | | [GGUF](https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/resolve/main/gemma-4-12b-it-apostate.i1-IQ1_S.gguf) | i1-IQ1_S | 3.4 | for the desperate | | [GGUF](https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/resolve/main/gemma-4-12b-it-apostate.i1-IQ1_M.gguf) | i1-IQ1_M | 3.6 | mostly desperate | | [GGUF](https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/resolve/main/gemma-4-12b-it-apostate.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.0 | | | [GGUF](https://huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF/resolve/main/gemma-4-12b-it-apostate.i1-IQ2_XS.gguf) | i1-IQ2_XS | 4.3 | | | [GGUF](https://huggingface.co/mrade...

Note3. Okt.huggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF

Source context: 1458 downloads · 1 likes · Library transformers · Repo mradermacher/gemma-4-12b-it-apostate-i1-GGUF

Note17. Julihuggingface.co/mradermacher/gemma-4-12b-it-apostate-i1-GGUF

Source context: 4097 downloads · 0 likes · Library transformers · Repo mradermacher/gemma-4-12b-it-apostate-i1-GGUF