Aller au contenu
ModelPrésentéExterne
Télécharger

Qwen2.5 0.5B Instruct

This repository contains a pre-compiled build of Qwen/Qwen2.5-0.5B-Instruct for running it on FuriosaAI RNGD with Furiosa-LLM.

Sourcehuggingface.co/furiosa-ai/Qwen2.5-0.5B-Instruct/tree/ae03de7e713961fd31b7405381f99481b33231d5
TypeText modelModèle de baseQwen/Qwen2.5-0.5B-InstructVersionmodel

Source du modèle

Type
Text model
Taille du fichier
950 MB
Modèle de base
Qwen/Qwen2.5-0.5B-Instruct
Version
vmodel
Mis à jour
25 oct. 2026
Téléchargements
714
Source
Hugging Face

Description de la source

Qwen2.5-0.5B-Instruct

This repository contains a pre-compiled build of Qwen/Qwen2.5-0.5B-Instruct for running it on FuriosaAI RNGD with Furiosa-LLM.

Overview

Qwen2.5-0.5B-Instruct is the smallest instruction-tuned model in the Qwen2.5 series, an auto-regressive dense transformer with about 0.5B parameters. Despite its size it offers solid instruction following, multilingual coverage, and tool usage, making it well suited to lightweight and latency-sensitive deployments. Its intended use is the same as the upstream Qwen/Qwen2.5-0.5B-Instruct, and it is released under the Apache 2.0 License.

  • Architecture: Qwen2 (dense)
  • Input / Output: Text / Text
  • Supported Inference Engine: Furiosa LLM
  • Supported Hardware: FuriosaAI RNGD

Quantization

No quantization — the model runs in its native 16-bit precision.

Features

  • Tool calling. The model supports tool (function) calling through the hermes tool-call parser, the parser used by the Qwen series.

Parallelism Strategy

On RNGD, Qwen2.5-0.5B-Instruct runs with a tensor-parallel size of 4 PEs, which fits on a single RNGD card (8 PEs per card).

Usage

To run this model with Furiosa-LLM, follow the example commands below after installing Furiosa-LLM and its prerequisites.

Launch the server

The simplest way to serve the model is:

# Launch the server, listening on port 8000 by default
furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct

To also enable tool (function) calling, add the hermes tool-call parser (the parser used by the Qwen series):

furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

When the server is ready, you will see:

INFO:     Started server process [27507]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Basic Usage

The server exposes an OpenAI-compatible API. You can send a request with curl:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "furiosa-ai/Qwen2.5-0.5B-Instruct",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
    }' \
    | python -m json.tool

Advanced Usage

Tool calling. With the server launched using --enable-auto-tool-choice --tool-call-parser hermes (see Launch the server), pass tools in the request and let the model decide when to call them. See the Tool Calling guide for a complete client example and details on tool-choice options.

Learn more

Sources

1 source
Hugging FacePrincipalVersion ae03de7e713961fd31b7405381f99481b33231d5
huggingface.co/furiosa-ai/Qwen2.5-0.5B-Instruct/tree/ae03de7e713961fd31b7405381f99481b33231d5

Vérifié 25 oct.

Artefacts du modèle

1 artefact

params-Qwen2.5-0.5B-Instruct-qwen2-24L-W16A16KV16-shard_size=5000000000-468ddb3c2472c3443ecb71e0f86cd692c5ac3b1ba5fed525c8585fa6004f8db1.safetensors/model.safetensors

safetensors · 950 MB · SHA-256 a5cd73b8d89e…ddc6 · Hugging Face

Télécharger

Extraits de sources

3 extraits
Readme17 juil.huggingface.co/furiosa-ai/Qwen2.5-0.5B-Instruct

--- base_model: Qwen/Qwen2.5-0.5B-Instruct language: - en license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE pipeline_tag: text-generation library_name: furiosa-llm tags: - furiosa-ai - qwen - qwen-2.5 --- # Qwen2.5-0.5B-Instruct This repository contains a pre-compiled build of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for running it on [FuriosaAI RNGD](https://furiosa.ai) with [Furiosa-LLM](https://developer.furiosa.ai/latest/en/furiosa_llm/intro.html). ## Overview Qwen2.5-0.5B-Instruct is the smallest instruction-tuned model in the Qwen2.5 series, an auto-regressive dense transformer with about 0.5B parameters. Despite its size it offers solid instruction following, multilingual coverage, and tool usage, making it well suited to lightweight and latency-sensitive deployments. Its intended use is the same as the upstream [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct), and it is released under the [Apache 2.0 License](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE). - **Architecture:** Qwen2 (dense) - **Input / Output:** Text / Text - **Supported Inference Engine:** Furiosa LLM - **Supported Hardware:** FuriosaAI RNGD ### Quantization No quantization — the model runs in its native 16-bit precision. ### Features - **Tool calling.** The model supports tool (function) calling through the `hermes` tool-call parser, the parser used by the Qwen series. ### Parallelism Strategy On RNGD, Qwen2.5-0.5B-Instruct runs with a **tensor-parallel size of 4 PEs**, which fits on a **single RNGD card** (8 PEs per card). ## Usage To run this model with Furiosa-LLM, follow the example commands below after [installing Furiosa-LLM and its prerequisites](https://developer.furiosa.ai/latest/en/get_started/furiosa_llm.html#installing-furiosa-llm). ### Launch the server The simplest way to serve the model is: ```sh # Launch the server, listening on port 8000 by default furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct ``` To also enable tool (function) calling, add the `hermes` tool-call parser (the parser used by the Qwen series): ```sh furiosa-llm serve furiosa-ai/Qwen2.5-0.5B-Instruct \ --enable-auto-tool-choice \ --tool-call-parser hermes ``` When the server is ready, you will see: ```sh INFO: Started server proc...

Note25 oct.huggingface.co/furiosa-ai/Qwen2.5-0.5B-Instruct

Source context: 714 downloads · 0 likes · Pipeline text-generation · Library furiosa-llm · Repo furiosa-ai/Qwen2.5-0.5B-Instruct

Note7 aoûthuggingface.co/furiosa-ai/Qwen2.5-0.5B-Instruct

Source context: 1939 downloads · 0 likes · Pipeline text-generation · Library furiosa-llm · Repo furiosa-ai/Qwen2.5-0.5B-Instruct