Aller au contenu
ModelVérifiéExterne
Browse files

Qwen2.5 Coder 14B Instruct Heretic I1 GGUF

For a convenient overview and download list, visit our model page for this model.

Sourcehuggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF/tree/5ef1835bac378901ff74e7c6c014895f36e800f8
TypeOtherModèle de baserichardyoung/Qwen2.5-Coder-14B-Instruct-hereticVersionRevision 5ef1835Éditeur@mradermacherCClasse

Source du modèle

Type
Other
Taille du fichier
8,21 MB–11,3 GB
Modèle de base
richardyoung/Qwen2.5-Coder-14B-Instruct-heretic
Version
vRevision 5ef1835
Mis à jour
23 oct. 2026
Éditeur
@mradermacher
Téléchargements
811
Paramètres
14B
Source
Hugging Face

Description de la source

About

weighted/imatrix quants of https://huggingface.co/richardyoung/Qwen2.5-Coder-14B-Instruct-heretic

For a convenient overview and download list, visit our model page for this model.

static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-GGUF

Usage

If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

LinkTypeSize/GBNotes
GGUFimatrix0.1imatrix file (for creating your own quants)
GGUFi1-IQ1_S3.7for the desperate
GGUFi1-IQ1_M4.0mostly desperate
GGUFi1-IQ2_XXS4.4
GGUFi1-IQ2_XS4.8
GGUFi1-IQ2_S5.1
GGUFi1-IQ2_M5.5
GGUFi1-Q2_K_S5.5very low quality
GGUFi1-Q2_K5.9IQ3_XXS probably better
GGUFi1-IQ3_XXS6.0lower quality
GGUFi1-IQ3_XS6.5
GGUFi1-Q3_K_S6.8IQ3_XS probably better
GGUFi1-IQ3_S6.8beats Q3_K*
GGUFi1-IQ3_M7.0
GGUFi1-Q3_K_M7.4IQ3_S probably better
GGUFi1-Q3_K_L8.0IQ3_M probably better
GGUFi1-IQ4_XS8.2
GGUFi1-Q4_08.6fast, low quality
GGUFi1-IQ4_NL8.6prefer IQ4_XS
GGUFi1-Q4_K_S8.7optimal size/speed/quality
GGUFi1-Q4_K_M9.1fast, recommended
GGUFi1-Q4_19.5
GGUFi1-Q5_K_S10.4
GGUFi1-Q5_K_M10.6
GGUFi1-Q6_K12.2practically like static Q6_K

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

image.png

And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9

FAQ / Model Request

See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.

Thanks

I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.

Sources

1 source
Hugging FacePrincipalVersion 5ef1835bac378901ff74e7c6c014895f36e800f8
huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF/tree/5ef1835bac378901ff74e7c6c014895f36e800f8

Vérifié 23 oct.

Download files

Artefacts du modèle

25 artefacts

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ1_M.gguf

gguf · 3,61 GB · SHA-256 841443014fc5…14a8 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ1_S.gguf

gguf · 3,36 GB · SHA-256 82c5943b7822…f7ba · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ2_M.gguf

gguf · 4,99 GB · SHA-256 68aa9c25e787…eb99 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ2_S.gguf

gguf · 4,66 GB · SHA-256 be3cab71df80…f672 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ2_XS.gguf

gguf · 4,38 GB · SHA-256 02f289926e6d…b13f · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ2_XXS.gguf

gguf · 4,02 GB · SHA-256 14c791caa588…cd74 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ3_M.gguf

gguf · 6,44 GB · SHA-256 310d9ca7fcb1…7a73 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ3_S.gguf

gguf · 6,23 GB · SHA-256 37e1ec24ba8a…20f3 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ3_XS.gguf

gguf · 5,94 GB · SHA-256 3471ca11eeea…d870 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ3_XXS.gguf

gguf · 5,54 GB · SHA-256 92c79e6bdb2a…df70 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ4_NL.gguf

gguf · 7,96 GB · SHA-256 4d87c4a028fe…7d75 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ4_XS.gguf

gguf · 7,56 GB · SHA-256 43cadf54efbe…9200 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q2_K_S.gguf

gguf · 5,03 GB · SHA-256 67830e6649cf…7c04 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q2_K.gguf

gguf · 5,37 GB · SHA-256 df47dec4d75e…9c22 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q3_K_L.gguf

gguf · 7,38 GB · SHA-256 b9f0eac7f6f4…e399 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q3_K_M.gguf

gguf · 6,84 GB · SHA-256 b245e6c6e3dc…2464 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q3_K_S.gguf

gguf · 6,20 GB · SHA-256 bf4cd21a1f8e…b606 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q4_0.gguf

gguf · 7,96 GB · SHA-256 ff99fad802b9…ec83 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q4_1.gguf

gguf · 8,75 GB · SHA-256 961d3883a264…a28c · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q4_K_M.gguf

gguf · 8,37 GB · SHA-256 32a404101509…84f1 · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q4_K_S.gguf

gguf · 7,98 GB · SHA-256 a47a772a0181…e8cd · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q5_K_M.gguf

gguf · 9,79 GB · SHA-256 25c7da325853…a42e · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q5_K_S.gguf

gguf · 9,56 GB · SHA-256 a33b6e6de6d7…a7ef · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.i1-Q6_K.gguf

gguf · 11,3 GB · SHA-256 35cdd04935b7…a49d · Hugging Face

Download file

Qwen2.5-Coder-14B-Instruct-heretic.imatrix.gguf

gguf · 8,21 MB · SHA-256 f7a60ac02ea5…e9db · Hugging Face

Download file

Extraits de sources

3 extraits
Readme11 juil.huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF

--- base_model: richardyoung/Qwen2.5-Coder-14B-Instruct-heretic language: - en library_name: transformers license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/blob/main/LICENSE mradermacher: readme_rev: 1 quantized_by: mradermacher tags: - code - codeqwen - chat - qwen - qwen-coder - heretic - uncensored - decensored - abliterated - reproducible --- ## About <!-- ### quantize_version: 2 --> <!-- ### output_tensor_quantised: 1 --> <!-- ### convert_type: hf --> <!-- ### vocab_type: --> <!-- ### tags: nicoboss --> <!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S --> <!-- ### quants_skip: --> <!-- ### skip_mmproj: --> weighted/imatrix quants of https://huggingface.co/richardyoung/Qwen2.5-Coder-14B-Instruct-heretic <!-- provided-files --> ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF).*** static quants are available at https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-GGUF ## Usage If you are unsure how to use GGUF files, refer to one of [TheBloke's READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for more details, including on how to concatenate multi-part files. ## Provided Quants (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-14B-Instruct-heretic.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ1_S.gguf) | i1-IQ1_S | 3.7 | for the desperate | | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ1_M.gguf) | i1-IQ1_M | 4.0 | mostly desperate | | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF/resolve/main/Qwen2.5-Coder-14B-Instruct-heretic.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 4.4 | | | [GGUF](https://huggingface.co/m...

Note23 oct.huggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF

Source context: 811 downloads · 1 likes · Library transformers · Repo mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF

Note6 aoûthuggingface.co/mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF

Source context: 617 downloads · 1 likes · Library transformers · Repo mradermacher/Qwen2.5-Coder-14B-Instruct-heretic-i1-GGUF