Loading
Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
For a convenient overview and download list, visit our model page for this model.
Model source
Source description
weighted/imatrix quants of https://huggingface.co/prithivMLmods/Qwen2-VL-2B-Abliterated-Caption-it
For a convenient overview and download list, visit our model page for this model.
static quants are available at
Sources
1 sourceVerified Sep 17
Model artifacts
25 artifactsSource excerpts
2 excerptsThis is a vision model - mmproj files (if any) will be in the static repository.
If you are unsure how to use GGUF files, refer to one of TheBloke's READMEs for more details, including on how to concatenate multi-part files.
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|---|---|---|---|
| GGUF | imatrix | 0.1 | imatrix file (for creating your own quants) |
| GGUF | i1-IQ1_S | 0.5 | for the desperate |
| GGUF | i1-IQ1_M | 0.6 | mostly desperate |
| GGUF | i1-IQ2_XXS | 0.6 |
Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):
image.png
And here are Artefact2's thoughts on the matter: https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
See https://huggingface.co/mradermacher/model_requests for some answers to questions you might have and/or if you want some other model quantized.
I thank my company, nethype GmbH, for letting me use its servers and providing upgrades to my workstation to enable this work in my free time. Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
Qwen2-VL-2B-Abliterated-Caption-it.i1-IQ2_M.gguf
gguf · 573 MB · SHA-256 cfe555867830…d38c · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ2_S.gguf
gguf · 538 MB · SHA-256 940552fbde10…876b · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ2_XS.gguf
gguf · 525 MB · SHA-256 4138c9834e98…54d1 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ2_XXS.gguf
gguf · 487 MB · SHA-256 584e7b11052b…d0ac · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ3_M.gguf
gguf · 741 MB · SHA-256 5fe33f380e15…e1b9 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ3_S.gguf
gguf · 727 MB · SHA-256 68a0b3ebc856…88f0 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ3_XS.gguf
gguf · 698 MB · SHA-256 b62cffd7d704…2401 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ3_XXS.gguf
gguf · 638 MB · SHA-256 07af044698a4…1e89 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ4_NL.gguf
gguf · 893 MB · SHA-256 afaa936fac99…eaf6 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-IQ4_XS.gguf
gguf · 854 MB · SHA-256 7b5374b9ab8e…6f61 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q2_K_S.gguf
gguf · 610 MB · SHA-256 e7a6eb064bda…9a5f · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q2_K.gguf
gguf · 645 MB · SHA-256 d7731017c0db…5e45 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q3_K_L.gguf
gguf · 839 MB · SHA-256 6825525e0d90…934c · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q3_K_M.gguf
gguf · 786 MB · SHA-256 8185fe9ba218…6bbd · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q3_K_S.gguf
gguf · 726 MB · SHA-256 d1311fcdb016…848e · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q4_0.gguf
gguf · 894 MB · SHA-256 e33c17197b39…1675 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q4_1.gguf
gguf · 970 MB · SHA-256 598b9cae1335…6d90 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q4_K_M.gguf
gguf · 940 MB · SHA-256 83760a1d2bdc…be1b · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q4_K_S.gguf
gguf · 897 MB · SHA-256 f3e7cf203280…788b · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q5_K_M.gguf
gguf · 1.05 GB · SHA-256 75e9f22a55f2…7934 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q5_K_S.gguf
gguf · 1.02 GB · SHA-256 a92df1429581…05a5 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.i1-Q6_K.gguf
gguf · 1.19 GB · SHA-256 de2d3c53a872…9ab6 · Hugging Face
DownloadQwen2-VL-2B-Abliterated-Caption-it.imatrix.gguf
gguf · 1.97 MB · SHA-256 d05b4140be96…3eb2 · Hugging Face
Download--- base_model: prithivMLmods/Qwen2-VL-2B-Abliterated-Caption-it datasets: - prithivMLmods/Caption3o-XL-v4 - prithivMLmods/Caption3o-Opt-v3-Tiny - prithivMLmods/Caption3o-LongCap-v4 - prithivMLmods/Caption3o-Opt-v3 language: - en library_name: transformers license: apache-2.0 mradermacher: readme_rev: 1 quantized_by: mradermacher tags: - text-generation-inference - uncensored - image-captioning - vlm - visual-understanding - caption - image-to-text --- ## About <!-- ### quantize_version: 2 --> <!-- ### output_tensor_quantised: 1 --> <!-- ### convert_type: hf --> <!-- ### vocab_type: --> <!-- ### tags: nicoboss --> <!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S --> <!-- ### quants_skip: --> <!-- ### skip_mmproj: --> weighted/imatrix quants of https://huggingface.co/prithivMLmods/Qwen2-VL-2B-Abliterated-Caption-it <!-- provided-files --> ***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen2-VL-2B-Abliterated-Caption-it-i1-GGUF).*** static quants are available at https://huggingface.co/mradermacher/Qwen2-VL-2B-Abliterated-Caption-it-GGUF **This is a vision model - mmproj files (if any) will be in the [static repository](https://huggingface.co/mradermacher/Qwen2-VL-2B-Abliterated-Caption-it-GGUF).** ## Usage If you are unsure how to use GGUF files, refer to one of [TheBloke's READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for more details, including on how to concatenate multi-part files. ## Provided Quants (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | [GGUF](https://huggingface.co/mradermacher/Qwen2-VL-2B-Abliterated-Caption-it-i1-GGUF/resolve/main/Qwen2-VL-2B-Abliterated-Caption-it.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) | | [GGUF](https://huggingface.co/mradermacher/Qwen2-VL-2B-Abliterated-Caption-it-i1-GGUF/resolve/main/Qwen2-VL-2B-Abliterated-Caption-it.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate | | [GGUF](https://huggingface.co/mradermacher/Qwen2-VL-2B-Abliterated-Caption-it-i1-GGUF/resolve/main/Qwen2-VL-2B-Abliterated-Caption-it.i1-IQ1_M.gguf) | i1-IQ1_M |...
Source context: 1005 downloads · 0 likes · Pipeline image-to-text · Library transformers · Repo mradermacher/Qwen2-VL-2B-Abliterated-Caption-it-i1-GGUF
| GGUF | i1-IQ2_XS | 0.7 |
| GGUF | i1-IQ2_S | 0.7 |
| GGUF | i1-IQ2_M | 0.7 |
| GGUF | i1-Q2_K_S | 0.7 | very low quality |
| GGUF | i1-IQ3_XXS | 0.8 | lower quality |
| GGUF | i1-Q2_K | 0.8 | IQ3_XXS probably better |
| GGUF | i1-IQ3_XS | 0.8 |
| GGUF | i1-Q3_K_S | 0.9 | IQ3_XS probably better |
| GGUF | i1-IQ3_S | 0.9 | beats Q3_K* |
| GGUF | i1-IQ3_M | 0.9 |
| GGUF | i1-Q3_K_M | 0.9 | IQ3_S probably better |
| GGUF | i1-Q3_K_L | 1.0 | IQ3_M probably better |
| GGUF | i1-IQ4_XS | 1.0 |
| GGUF | i1-IQ4_NL | 1.0 | prefer IQ4_XS |
| GGUF | i1-Q4_0 | 1.0 | fast, low quality |
| GGUF | i1-Q4_K_S | 1.0 | optimal size/speed/quality |
| GGUF | i1-Q4_K_M | 1.1 | fast, recommended |
| GGUF | i1-Q4_1 | 1.1 |
| GGUF | i1-Q5_K_S | 1.2 |
| GGUF | i1-Q5_K_M | 1.2 |
| GGUF | i1-Q6_K | 1.4 | practically like static Q6_K |
