ModelPublicadoExterno

Qwen3 8B Q5 K M

Qwen3-8B in Q5KM GGUF format for CPU/GPU inference with llama.cpp or llama-cpp-python.

Fuentehuggingface.co/Mukilan06/Qwen3-8B-Q5_K_M/tree/ea0345f6053c552b9e22d262423d24856e7df0f1

TipoOtherModelo baseQwen/Qwen3-8BVersiónQwen3 8 B Q5 K MBcalidad

Fuente del modelo

Tipo: Other
Tamaño del archivo: 5,45 GB
Modelo base: Qwen/Qwen3-8B
Versión: vQwen3 8 B Q5 K M
Actualizado: 18 oct 2026
Fuente: Hugging Face

Descripción de la fuente

Qwen3-8B-Q5_K_M

Qwen3-8B in Q5_K_M GGUF format for CPU/GPU inference with llama.cpp or llama-cpp-python.

Source base model: Qwen/Qwen3-8B
Original GGUF source: Qwen/Qwen3-8B-GGUF
Quantization: Q5_K_M (~5.85 GB)
License: Apache 2.0

Usage

py -3.11 -m pip install llama-cpp-python transformers

from llama_cpp import Llama
from transformers import AutoTokenizer

llm = Llama(model_path="Qwen3-8B-Q5_K_M.gguf", n_ctx=8192)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

messages = [{"role": "user", "content": "Hello!"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
out = llm(prompt, max_tokens=256, stop=[tokenizer.eos_token])
print(out["choices"][0]["text"])

Fuentes

1 fuente

Hugging FacePrincipalVersión ea0345f6053c552b9e22d262423d24856e7df0f1

huggingface.co/Mukilan06/Qwen3-8B-Q5_K_M/tree/ea0345f6053c552b9e22d262423d24856e7df0f1

Verificado 18 oct

Artefactos del modelo

1 artefacto

Qwen3-8B-Q5_K_M.gguf

gguf · 5,45 GB · SHA-256 068bae163faa…2738 · Hugging Face

Download file

Extractos de fuentes

3 extractos

Readme8 agohuggingface.co/Mukilan06/Qwen3-8B-Q5_K_M

--- license: apache-2.0 base_model: Qwen/Qwen3-8B tags: - gguf - qwen3 - q5_k_m - text-generation library_name: llama-cpp-python --- # Qwen3-8B-Q5_K_M Qwen3-8B in **Q5_K_M** GGUF format for CPU/GPU inference with llama.cpp or llama-cpp-python. - Source base model: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) - Original GGUF source: [Qwen/Qwen3-8B-GGUF](https://huggingface.co/Qwen/Qwen3-8B-GGUF) - Quantization: Q5_K_M (~5.85 GB) - License: Apache 2.0 ## Usage ```powershell py -3.11 -m pip install llama-cpp-python transformers ``` ```python from llama_cpp import Llama from transformers import AutoTokenizer llm = Llama(model_path="Qwen3-8B-Q5_K_M.gguf", n_ctx=8192) tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B") messages = [{"role": "user", "content": "Hello!"}] prompt = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False ) out = llm(prompt, max_tokens=256, stop=[tokenizer.eos_token]) print(out["choices"][0]["text"]) ```

Note18 octhuggingface.co/Mukilan06/Qwen3-8B-Q5_K_M

Source context: 60 downloads · 0 likes · Pipeline text-generation · Library llama-cpp-python · Repo Mukilan06/Qwen3-8B-Q5_K_M

Note8 agohuggingface.co/Mukilan06/Qwen3-8B-Q5_K_M

Source context: 20 downloads · 0 likes · Pipeline text-generation · Library llama-cpp-python · Repo Mukilan06/Qwen3-8B-Q5_K_M