Qwen2.5 0.5B Instruct
This repository contains a pre-compiled build of Qwen/Qwen2.5-0.5B-Instruct for running it on FuriosaAI RNGD with Furiosa-LLM.
Modellquelle
- Art
- Text model
- Dateigröße
- 950 MB
- Basismodell
- Qwen/Qwen2.5-0.5B-Instruct
- Version
- vmodel
- Aktualisiert
- 25. Okt. 2026
- Downloads
- 714
- Quelle
- Hugging Face
Quellenbeschreibung
Qwen2.5-0.5B-Instruct
This repository contains a pre-compiled build of Qwen/Qwen2.5-0.5B-Instruct for running it on FuriosaAI RNGD with Furiosa-LLM.
Overview
Qwen2.5-0.5B-Instruct is the smallest instruction-tuned model in the Qwen2.5 series, an auto-regressive dense transformer with about 0.5B parameters. Despite its size it offers solid instruction following, multilingual coverage, and tool usage, making it well suited to lightweight and latency-sensitive deployments. Its intended use is the same as the upstream Qwen/Qwen2.5-0.5B-Instruct, and it is released under the Apache 2.0 License.
- Architecture: Qwen2 (dense)
- Input / Output: Text / Text
- Supported Inference Engine: Furiosa LLM
- Supported Hardware: FuriosaAI RNGD
Quantization
No quantization — the model runs in its native 16-bit precision.
Features
- Tool calling. The model supports tool (function) calling through the
hermestool-call parser, the parser used by the Qwen series.
Parallelism Strategy
On RNGD, Qwen2.5-0.5B-Instruct runs with a tensor-parallel size of 4 PEs, which fits on a single RNGD card (8 PEs per card).
Usage
To run this model with Furiosa-LLM, follow the example commands below after installing Furiosa-LLM and its prerequisites.
Launch the server
The simplest way to serve the model is:
# Launch the server, listening on port 8000 by default
furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct
To also enable tool (function) calling, add the hermes tool-call parser
(the parser used by the Qwen series):
furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct \
--enable-auto-tool-choice \
--tool-call-parser hermes
When the server is ready, you will see:
INFO: Started server process [27507]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Basic Usage
The server exposes an OpenAI-compatible API. You can send a request with curl:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "furiosa-ai/Qwen2.5-0.5B-Instruct",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}' \
| python -m json.tool
Advanced Usage
Tool calling. With the server launched using
--enable-auto-tool-choice --tool-call-parser hermes (see
Launch the server), pass tools in the request and let the
model decide when to call them. See the
Tool Calling guide
for a complete client example and details on tool-choice options.
Learn more
- Tool Calling — parsers, tool-choice options, and more examples
- Furiosa-LLM Server (
furiosa-llm serve) — full OpenAI-compatible API reference and serving options - Qwen/Qwen2.5-0.5B-Instruct — upstream model card
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 25. Okt.
Modellartefakte
1 Artefakt
Modellartefakte
1 Artefaktparams-Qwen2.5-0.5B-Instruct-qwen2-24L-W16A16KV16-shard_size=5000000000-468ddb3c2472c3443ecb71e0f86cd692c5ac3b1ba5fed525c8585fa6004f8db1.safetensors/model.safetensors
safetensors · 950 MB · SHA-256 a5cd73b8d89e…ddc6 · Hugging Face
HerunterladenQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- base_model: Qwen/Qwen2.5-0.5B-Instruct language: - en license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE pipeline_tag: text-generation library_name: furiosa-llm tags: - furiosa-ai - qwen - qwen-2.5 --- # Qwen2.5-0.5B-Instruct This repository contains a pre-compiled build of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for running it on [FuriosaAI RNGD](https://furiosa.ai) with [Furiosa-LLM](https://developer.furiosa.ai/latest/en/furiosa_llm/intro.html). ## Overview Qwen2.5-0.5B-Instruct is the smallest instruction-tuned model in the Qwen2.5 series, an auto-regressive dense transformer with about 0.5B parameters. Despite its size it offers solid instruction following, multilingual coverage, and tool usage, making it well suited to lightweight and latency-sensitive deployments. Its intended use is the same as the upstream [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), and it is released under the [Apache 2.0 License](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE). - **Architecture:** Qwen2 (dense) - **Input / Output:** Text / Text - **Supported Inference Engine:** Furiosa LLM - **Supported Hardware:** FuriosaAI RNGD ### Quantization No quantization — the model runs in its native 16-bit precision. ### Features - **Tool calling.** The model supports tool (function) calling through the `hermes` tool-call parser, the parser used by the Qwen series. ### Parallelism Strategy On RNGD, Qwen2.5-0.5B-Instruct runs with a **tensor-parallel size of 4 PEs**, which fits on a **single RNGD card** (8 PEs per card). ## Usage To run this model with Furiosa-LLM, follow the example commands below after [installing Furiosa-LLM and its prerequisites](https://developer.furiosa.ai/latest/en/get_started/furiosa_llm.html#installing-furiosa-llm). ### Launch the server The simplest way to serve the model is: ```sh # Launch the server, listening on port 8000 by default furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct ``` To also enable tool (function) calling, add the `hermes` tool-call parser (the parser used by the Qwen series): ```sh furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct \ --enable-auto-tool-choice \ --tool-call-parser hermes ``` When the server is ready, you will see: ```sh INFO: Started server proc...
Source context: 714 downloads · 0 likes · Pipeline text-generation · Library furiosa-llm · Repo furiosa-ai/Qwen2.5-0.5B-Instruct
Source context: 1939 downloads · 0 likes · Pipeline text-generation · Library furiosa-llm · Repo furiosa-ai/Qwen2.5-0.5B-Instruct
