ModelListadosExterno

Qwen3 30B A3B FP8

This repository contains Qwen/Qwen3-30B-A3B-FP8 together with a Furiosa Executable Bundle (FXB) for running it on FuriosaAI RNGD with Furiosa-LLM. The same model also runs on other frameworks (such as vLLM, SGLang,...

Fontehuggingface.co/furiosa-ai/Qwen3-30B-A3B-FP8/tree/8b412055cfce2a6824148b22ba0c8ae6150e54c4

TipoText modelModelo baseQwen/Qwen3-30B-A3B-FP8VersãoRevision 8b41205Bnota

Fonte do modelo

Tipo: Text model
Tamanho do arquivo: Ainda não registrado
Modelo base: Qwen/Qwen3-30B-A3B-FP8
Versão: vRevision 8b41205
Atualizado: 23 de out. de 2026
Downloads: 794
Fonte: Hugging Face

Descrição da fonte

Qwen3-30B-A3B-FP8

Overview

Qwen3-30B-A3B is an auto-regressive Mixture-of-Experts (MoE) transformer with 30.5B total parameters of which about 3.3B are activated per token. It is a hybrid reasoning model that supports both thinking and non-thinking modes within a single model — switchable per request — and offers strong reasoning over text, instruction following, multilingual coverage, and tool usage. Its intended use is the same as the upstream Qwen/Qwen3-30B-A3B-FP8, and it is released under the Apache 2.0 License.

Architecture: Qwen3-MoE (Mixture-of-Experts)
Input / Output: Text / Text
Supported Inference Engine: Furiosa LLM
Supported Hardware: FuriosaAI RNGD

Quantization

Weights are quantized to FP8 (static), following the upstream FP8 release, and activations use dynamic FP8 quantization at runtime (per-token / per-block). The KV cache stays in 16-bit precision.

Features

Reasoning. Qwen3-30B-A3B is a hybrid reasoning model: thinking mode is on by default and can be toggled per request via enable_thinking. Launch the server with --reasoning-parser qwen3 to have Furiosa-LLM parse the chain of thought into a separate field (see Basic Usage below).
Tool calling. The model supports tool (function) calling through the hermes tool-call parser, the parser used by the Qwen3 series.

Parallelism Strategy

On RNGD, Qwen3-30B-A3B-FP8 runs with a tensor-parallel size of 32 PEs, which maps to four RNGD cards (8 PEs per card).

Usage

To run this model with Furiosa-LLM, follow the example commands below after installing Furiosa-LLM and its prerequisites.

Launch the server

Qwen3-30B-A3B-FP8 is a hybrid model that reasons by default and can switch thinking on and off (see Advanced Usage). Serve it with the qwen3 reasoning parser so the chain of thought is returned in a separate field:

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-FP8 \
  --reasoning-parser qwen3

To also enable tool (function) calling, add the hermes tool-call parser (the parser used by the Qwen3 series):

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-FP8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

When the server is ready, you will see:

INFO:     Started server process [27507]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Basic Usage

The server exposes an OpenAI-compatible API. You can send a request with curl:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "furiosa-ai/Qwen3-30B-A3B-FP8",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
    }' \
    | python -m json.tool

In thinking mode, Qwen3-30B-A3B-FP8 returns its reasoning separately from the final answer:

response.choices[].message.reasoning (non-streaming)
response.choices[].delta.reasoning (streaming)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="furiosa-ai/Qwen3-30B-A3B-FP8",
    messages=[{"role": "user", "content": "How many r's are in 'strawberry'?"}],
)

print("Reasoning:", response.choices[0].message.reasoning)
print("Answer:", response.choices[0].message.content)

Note: The reasoning field is not part of the OpenAI API specification but is a widely followed convention (the OpenAI Agents SDK, vLLM, and others). It appears only in responses that contain reasoning content; accessing it otherwise raises an AttributeError.

Advanced Usage

Toggling thinking. Qwen3-30B-A3B-FP8 reasons by default. To turn thinking off for a single request, pass enable_thinking through chat_template_kwargs; the response then carries no reasoning content, so read only message.content:

# Disable thinking for a single request
response = client.chat.completions.create(
    model="furiosa-ai/Qwen3-30B-A3B-FP8",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)

To default every request to non-thinking, launch the server with --default-chat-template-kwargs (a request can still re-enable thinking with its own chat_template_kwargs):

furiosa-llm serve furiosa-ai/Qwen3-30B-A3B-FP8 \
  --reasoning-parser qwen3 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

Tool calling. With the server launched using --enable-auto-tool-choice --tool-call-parser hermes (see Launch the server), pass tools in the request and let the model decide when to call them. See the Tool Calling guide for a complete client example and details on tool-choice options.

Learn more

Tool Calling — parsers, tool-choice options, and more examples
Furiosa-LLM Server (furiosa-llm serve) — full OpenAI-compatible API reference and serving options
Qwen/Qwen3-30B-A3B-FP8 — upstream model card

Fontes

1 fonte

Hugging FacePrimárioVersão 8b412055cfce2a6824148b22ba0c8ae6150e54c4

huggingface.co/furiosa-ai/Qwen3-30B-A3B-FP8/tree/8b412055cfce2a6824148b22ba0c8ae6150e54c4

Verificado 23 de out.

Trechos de fonte

3 trechos

Readme23 de out.huggingface.co/furiosa-ai/Qwen3-30B-A3B-FP8

--- base_model: Qwen/Qwen3-30B-A3B-FP8 license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8/blob/main/LICENSE pipeline_tag: text-generation library_name: furiosa-llm tags: - furiosa-ai - qwen3 - qwen3_moe - fp8 --- # Qwen3-30B-A3B-FP8 This repository contains [Qwen/Qwen3-30B-A3B-FP8](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8) together with a Furiosa Executable Bundle (FXB) for running it on [FuriosaAI RNGD](https://furiosa.ai) with [Furiosa-LLM](https://developer.furiosa.ai/latest/en/furiosa_llm/intro.html). The same model also runs on other frameworks (such as vLLM, SGLang, and Transformers); for usage with those, see the upstream [Qwen/Qwen3-30B-A3B-FP8](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8) model card. ## Overview Qwen3-30B-A3B is an auto-regressive **Mixture-of-Experts (MoE)** transformer with 30.5B total parameters of which about 3.3B are activated per token. It is a hybrid reasoning model that supports both thinking and non-thinking modes within a single model — switchable per request — and offers strong reasoning over text, instruction following, multilingual coverage, and tool usage. Its intended use is the same as the upstream [Qwen/Qwen3-30B-A3B-FP8](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8), and it is released under the [Apache 2.0 License](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8/blob/main/LICENSE). - **Architecture:** Qwen3-MoE (Mixture-of-Experts) - **Input / Output:** Text / Text - **Supported Inference Engine:** Furiosa LLM - **Supported Hardware:** FuriosaAI RNGD ### Quantization Weights are quantized to **FP8** (static), following the upstream FP8 release, and activations use **dynamic FP8 quantization** at runtime (per-token / per-block). The KV cache stays in 16-bit precision. ### Features - **Reasoning.** Qwen3-30B-A3B is a hybrid reasoning model: thinking mode is on by default and can be toggled per request via `enable_thinking`. Launch the server with `--reasoning-parser qwen3` to have Furiosa-LLM parse the chain of thought into a separate field (see [Basic Usage](#basic-usage) below). - **Tool calling.** The model supports tool (function) calling through the `hermes` tool-call parser, the parser used by the Qwen3 series. ### Parallelism Strategy On RNGD, Qwen3-30B-A3B-FP8 runs with a **tensor-parallel size of 32 PEs**, which maps to **four RNGD cards** (8 PEs per card)...

Readme17 de jul.huggingface.co/furiosa-ai/Qwen3-30B-A3B-FP8

--- base_model: Qwen/Qwen3-30B-A3B-FP8 license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8/blob/main/LICENSE pipeline_tag: text-generation library_name: furiosa-llm tags: - furiosa-ai - qwen3 - qwen3_moe - fp8 --- # Qwen3-30B-A3B-FP8 This repository contains [Qwen/Qwen3-30B-A3B-FP8](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8) together with a Furiosa Executable Bundle (FXB) for running it on [FuriosaAI RNGD](https://furiosa.ai) with [Furiosa-LLM](https://developer.furiosa.ai/latest/en/furiosa_llm/intro.html). The same model also runs on other frameworks (such as vLLM, SGLang, and Transformers); for usage with those, see the upstream [Qwen/Qwen3-30B-A3B-FP8](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8) model card. ## Overview Qwen3-30B-A3B is an auto-regressive **Mixture-of-Experts (MoE)** transformer with 30.5B total parameters of which about 3.3B are activated per token. It is a hybrid reasoning model that supports both thinking and non-thinking modes within a single model — switchable per request — and offers strong reasoning over text, instruction following, multilingual coverage, and tool usage. Its intended use is the same as the upstream [Qwen/Qwen3-30B-A3B-FP8](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8), and it is released under the [Apache 2.0 License](https://huggingface.co/Qwen/Qwen3-30B-A3B-FP8/blob/main/LICENSE). - **Architecture:** Qwen3-MoE (Mixture-of-Experts) - **Input / Output:** Text / Text - **Supported Inference Engine:** Furiosa LLM - **Supported Hardware:** FuriosaAI RNGD ### Quantization Weights are quantized to **FP8** (static), following the upstream FP8 release, and activations use **dynamic FP8 quantization** at runtime (per-token / per-block). The KV cache stays in 16-bit precision. ### Features - **Reasoning.** Qwen3-30B-A3B is a hybrid reasoning model: thinking mode is on by default and can be toggled per request via `enable_thinking`. Launch the server with `--reasoning-parser qwen3` to have Furiosa-LLM parse the chain of thought into a separate field (see [Basic Usage](#basic-usage) below). - **Tool calling.** The model supports tool (function) calling through the `hermes` tool-call parser, the parser used by the Qwen3 series. ### Parallelism Strategy On RNGD, Qwen3-30B-A3B-FP8 runs with a **tensor-parallel size of 16 PEs**, which maps to **two RNGD cards** (8 PEs per card)....

Note23 de out.huggingface.co/furiosa-ai/Qwen3-30B-A3B-FP8

Source context: 794 downloads · 0 likes · Pipeline text-generation · Library furiosa-llm · Repo furiosa-ai/Qwen3-30B-A3B-FP8