Qwen3 Asr 0.6b Rapidspeech GGUF
Single-file GGUF builds of Qwen/Qwen3-ASR-0.6B (audio encoder + Qwen3-0.6B LLM decoder) that run with CUDA on the original Jetson Nano gen1 (Tegra X1, sm53, CUDA 10.2). Native Traditional Chinese + punctuation, zh/en...
Source du modèle
- Type
- Other
- Taille du fichier
- 515 MB–1,75 GB
- Modèle de base
- Qwen/Qwen3-ASR-0.6B
- Version
- vRevision afdbe99
- Mis à jour
- 20 oct. 2026
- Éditeur
- @Luigi
- Téléchargements
- 69
- Paramètres
- 6B
- Source
- Hugging Face
Description de la source
Qwen3-ASR-0.6B — GGUF (Jetson Nano gen1 optimized)
Single-file GGUF builds of Qwen/Qwen3-ASR-0.6B (audio encoder + Qwen3-0.6B LLM decoder) that run with CUDA on the original Jetson Nano gen1 (Tegra X1, sm_53, CUDA 10.2). Native Traditional Chinese + punctuation, zh/en code-switch.
Variants
| file | size | accuracy | recommended for |
|---|---|---|---|
qwen3-asr-0.6b-f16.gguf | 1.88 GB | full (native Traditional) | reference / max accuracy |
qwen3-asr-0.6b-q8_0.gguf | 0.96 GB | full (native Traditional) | Jetson Nano gen1 (recommended) |
qwen3-asr-0.6b-q4_0.gguf | 0.52 GB | warning: collapses to Simplified | Ampere+ GPUs only, Simplified-OK |
Why q8_0, not q4_0, on the Jetson Nano (Maxwell sm_53)
Maxwell sm_53 has no dp4a (int8 dot-product), so ggml dequantizes low-bit
weights and the extra ALU work makes q4_0 slower than q8_0/f16 on this GPU
(measured ~1.7x slower). q4_0 also loses Traditional-character precision on
this 0.6B model (output collapses to Simplified). So q8_0 is the sweet spot:
half the size of f16, same speed and accuracy on Maxwell. q4_0 is only worth it on
Ampere+ (fast low-bit GEMM) where Simplified output is acceptable.
Engines (both Jetson-Nano-gen1 optimized)
Two open engines run this GGUF on the Nano gen1; both share the cuFFT batched mel and the CUDA-10.2/sm_53 build fixes:
| engine | RTF (13.5s clip, q8_0) | notes |
|---|---|---|
| vieenrose/qwen3-asr.cpp (standalone) | ~1.1 | dedicated 2-engine design: encoder on GPU + autoregressive decode on CPU (~195 ms/tok); fastest on the Nano |
| vieenrose/RapidSpeech.cpp (integrated) | ~1.3 | Qwen3-ASR inside the multi-model RapidSpeech framework (all weights on GPU); ships the converter scripts/convert_qwen3_asr_to_gguf.py |
The standalone reaches lower RTF because it keeps encoder weights on the GPU and LLM weights on the CPU simultaneously (batch-1 decode is faster on the A57 than on Maxwell); RapidSpeech's unified single-weight-buffer keeps everything on GPU.
Usage
Standalone:
qwen3-asr-cli -m qwen3-asr-0.6b-q8_0.gguf -f clip_16k.wav -t 4
RapidSpeech:
rs-asr-offline -m qwen3-asr-0.6b-q8_0.gguf -w clip_16k.wav
Convert your own
From a local Qwen/Qwen3-ASR-0.6B checkpoint, using the RapidSpeech fork:
python scripts/convert_qwen3_asr_to_gguf.py --hf-dir /path/to/Qwen3-ASR-0.6B \
--output qwen3-asr-0.6b-q8_0.gguf --quant q8_0
Sources
1 source
Sources
1 sourceVérifié 20 oct.
Artefacts du modèle
3 artefacts
Artefacts du modèle
3 artefactsExtraits de sources
3 extraits
Extraits de sources
3 extraits--- license: apache-2.0 base_model: Qwen/Qwen3-ASR-0.6B tags: [automatic-speech-recognition, gguf, rapidspeech, qwen3-asr, jetson] language: [zh, en] library_name: gguf --- # Qwen3-ASR-0.6B — GGUF (Jetson Nano gen1 optimized) Single-file GGUF builds of **Qwen/Qwen3-ASR-0.6B** (audio encoder + Qwen3-0.6B LLM decoder) that run with CUDA on the original **Jetson Nano gen1** (Tegra X1, sm_53, CUDA 10.2). Native Traditional Chinese + punctuation, zh/en code-switch. ## Variants | file | size | accuracy | recommended for | |---|---|---|---| | `qwen3-asr-0.6b-f16.gguf` | 1.88 GB | full (native Traditional) | reference / max accuracy | | `qwen3-asr-0.6b-q8_0.gguf` | 0.96 GB | full (native Traditional) | **Jetson Nano gen1 (recommended)** | | `qwen3-asr-0.6b-q4_0.gguf` | 0.52 GB | warning: collapses to Simplified | Ampere+ GPUs only, Simplified-OK | ### Why q8_0, not q4_0, on the Jetson Nano (Maxwell sm_53) Maxwell sm_53 has **no `dp4a`** (int8 dot-product), so ggml dequantizes low-bit weights and the extra ALU work makes **q4_0 *slower* than q8_0/f16 on this GPU** (measured ~1.7x slower). q4_0 also **loses Traditional-character precision** on this 0.6B model (output collapses to Simplified). So **q8_0 is the sweet spot**: half the size of f16, same speed and accuracy on Maxwell. q4_0 is only worth it on Ampere+ (fast low-bit GEMM) where Simplified output is acceptable. ## Engines (both Jetson-Nano-gen1 optimized) Two open engines run this GGUF on the Nano gen1; both share the cuFFT batched mel and the CUDA-10.2/sm_53 build fixes: | engine | RTF (13.5s clip, q8_0) | notes | |---|---|---| | **[vieenrose/qwen3-asr.cpp](https://github.com/vieenrose/qwen3-asr.cpp/tree/jetson-nano-gen1)** (standalone) | **~1.1** | dedicated 2-engine design: encoder on GPU + autoregressive decode on CPU (~195 ms/tok); fastest on the Nano | | **[vieenrose/RapidSpeech.cpp](https://github.com/vieenrose/RapidSpeech.cpp/tree/jetson-nano-gen1)** (integrated) | **~1.3** | Qwen3-ASR inside the multi-model RapidSpeech framework (all weights on GPU); ships the converter `scripts/convert_qwen3_asr_to_gguf.py` | The standalone reaches lower RTF because it keeps encoder weights on the GPU and LLM weights on the CPU simultaneously (batch-1 decode is faster on the A57 than on Maxwell); RapidSpeech's unified single-weight-buffer keeps everything on GPU. ## Usage Standalone: ```sh qwen3-asr-c...
Source context: 69 downloads · 0 likes · Pipeline automatic-speech-recognition · Library gguf · Repo Luigi/qwen3-asr-0.6b-rapidspeech-gguf
Source context: 50 downloads · 0 likes · Pipeline automatic-speech-recognition · Library gguf · Repo Luigi/qwen3-asr-0.6b-rapidspeech-gguf
