Qwen3 Asr 0.6b Rapidspeech GGUF

--- license: apache-2.0 base_model: Qwen/Qwen3-ASR-0.6B tags: [automatic-speech-recognition, gguf, rapidspeech, qwen3-asr, jetson] language: [zh, en] library_name: gguf --- # Qwen3-ASR-0.6B — GGUF (Jetson Nano gen1 optimized) Single-file GGUF builds of **Qwen/Qwen3-ASR-0.6B** (audio encoder + Qwen3-0.6B LLM decoder) that run with CUDA on the original **Jetson Nano gen1** (Tegra X1, sm_53, CUDA 10.2). Native Traditional Chinese + punctuation, zh/en code-switch. ## Variants | file | size | accuracy | recommended for | |---|---|---|---| | `qwen3-asr-0.6b-f16.gguf` | 1.88 GB | full (native Traditional) | reference / max accuracy | | `qwen3-asr-0.6b-q8_0.gguf` | 0.96 GB | full (native Traditional) | **Jetson Nano gen1 (recommended)** | | `qwen3-asr-0.6b-q4_0.gguf` | 0.52 GB | warning: collapses to Simplified | Ampere+ GPUs only, Simplified-OK | ### Why q8_0, not q4_0, on the Jetson Nano (Maxwell sm_53) Maxwell sm_53 has **no `dp4a`** (int8 dot-product), so ggml dequantizes low-bit weights and the extra ALU work makes **q4_0 *slower* than q8_0/f16 on this GPU** (measured ~1.7x slower). q4_0 also **loses Traditional-character precision** on this 0.6B model (output collapses to Simplified). So **q8_0 is the sweet spot**: half the size of f16, same speed and accuracy on Maxwell. q4_0 is only worth it on Ampere+ (fast low-bit GEMM) where Simplified output is acceptable. ## Engines (both Jetson-Nano-gen1 optimized) Two open engines run this GGUF on the Nano gen1; both share the cuFFT batched mel and the CUDA-10.2/sm_53 build fixes: | engine | RTF (13.5s clip, q8_0) | notes | |---|---|---| | **[vieenrose/qwen3-asr.cpp](https://github.com/vieenrose/qwen3-asr.cpp/tree/jetson-nano-gen1)** (standalone) | **~1.1** | dedicated 2-engine design: encoder on GPU + autoregressive decode on CPU (~195 ms/tok); fastest on the Nano | | **[vieenrose/RapidSpeech.cpp](https://github.com/vieenrose/RapidSpeech.cpp/tree/jetson-nano-gen1)** (integrated) | **~1.3** | Qwen3-ASR inside the multi-model RapidSpeech framework (all weights on GPU); ships the converter `scripts/convert_qwen3_asr_to_gguf.py` | The standalone reaches lower RTF because it keeps encoder weights on the GPU and LLM weights on the CPU simultaneously (batch-1 decode is faster on the A57 than on Maxwell); RapidSpeech's unified single-weight-buffer keeps everything on GPU. ## Usage Standalone: ```sh qwen3-asr-cli -m qwen3-asr-0.6b-q8_0.gguf -f clip_16k.wav -t 4 ``` RapidSpeech: ```sh rs-asr-offline -m qwen3-asr-0.6b-q8_0.gguf -w clip_16k.wav ``` ## Convert your own From a local `Qwen/Qwen3-ASR-0.6B` checkpoint, using the RapidSpeech fork: ```sh python scripts/convert_qwen3_asr_to_gguf.py --hf-dir /path/to/Qwen3-ASR-0.6B \ --output qwen3-asr-0.6b-q8_0.gguf --quant q8_0 ```