Parakeet Tdt 0.6b v2 Mlx 8bit
8-bit quantized MLX weights for NVIDIA Parakeet TDT 0.6B v2, ready to load on Apple Silicon without any quantization step at runtime.
Fuente del modelo
- Tipo
- Other
- Tamaño del archivo
- 890 MB
- Modelo base
- nvidia/parakeet-tdt-0.6b-v2
- Versión
- vRevision 6a7f872
- Actualizado
- 26 sept 2026
- Descargas
- 12
- Fuente
- Hugging Face
Descripción de la fuente
parakeet-tdt-0.6b-v2-mlx-8bit
8-bit quantized MLX weights for NVIDIA Parakeet TDT 0.6B v2, ready to load on Apple Silicon without any quantization step at runtime.
Published by CosmicFusion Labs for WhisperBud, an on-device dictation app for macOS.
Why this exists
The upstream MLX weights (mlx-community/parakeet-tdt-0.6b-v2) are
full precision: 2,472 MB to download, then quantized on each machine at
load time. These weights are the result of that same quantization
(mlx.nn.quantize, bits=8, group_size=64), saved once:
| upstream | this repo | |
|---|---|---|
| download | 2,472 MB | 933 MB |
| model load | ~1.7 s | ~0.05 s |
| transcription output | reference | identical |
Verified byte-identical transcripts against the quantize-at-load path on real speech clips (English, 5 to 45 seconds).
Usage
parakeet_mlx.from_pretrained() does NOT load quantized weights: it builds a
full-precision model and loads the tensors into it. Build the quantized
structure first, then load:
import json
import mlx.nn as nn
from huggingface_hub import hf_hub_download
from parakeet_mlx.utils import from_config
repo = "cosmicfusionlabs/parakeet-tdt-0.6b-v2-mlx-8bit"
config = json.load(open(hf_hub_download(repo, "config.json")))
quant = config.pop("quantization") # {"bits": 8, "group_size": 64}
model = from_config(config)
nn.quantize(model, bits=quant["bits"], group_size=quant["group_size"])
model.load_weights(hf_hub_download(repo, "model.safetensors"))
model.eval()
result = model.transcribe("audio.wav")
print(result.text)
Accuracy
Same weights as quantizing the upstream model at load time, so accuracy is
unchanged from the 8-bit path. In our own benchmark on real (non-synthetic)
English speech, 8-bit Parakeet TDT 0.6B v2 measured 11.6% mean WER, between
whisper-small.en (13.5%) and whisper-medium.en (8.2%), while running
2 to 4x faster than medium.en on an M1 Pro. 4-bit was measured too and
degraded accuracy (13.3% WER), which is why this repo ships 8-bit.
Credits and license
- Original model: nvidia/parakeet-tdt-0.6b-v2 (CC-BY-4.0)
- MLX conversion: mlx-community/parakeet-tdt-0.6b-v2
- Runtime: parakeet-mlx
Released under CC-BY-4.0, the same license as the original NVIDIA model.
Fuentes
1 fuente
Fuentes
1 fuenteVerificado 6 sept
Artefactos del modelo
1 artefacto
Artefactos del modelo
1 artefactoExtractos de fuentes
2 extractos
Extractos de fuentes
2 extractos--- license: cc-by-4.0 language: - en library_name: mlx tags: - automatic-speech-recognition - parakeet - mlx - quantized - apple-silicon base_model: nvidia/parakeet-tdt-0.6b-v2 pipeline_tag: automatic-speech-recognition --- # parakeet-tdt-0.6b-v2-mlx-8bit 8-bit quantized MLX weights for **NVIDIA Parakeet TDT 0.6B v2**, ready to load on Apple Silicon without any quantization step at runtime. Published by [CosmicFusion Labs](https://whisperbud.com) for [WhisperBud](https://whisperbud.com), an on-device dictation app for macOS. ## Why this exists The upstream MLX weights (`mlx-community/parakeet-tdt-0.6b-v2`) are full precision: **2,472 MB** to download, then quantized on each machine at load time. These weights are the result of that same quantization (`mlx.nn.quantize`, `bits=8`, `group_size=64`), saved once: | | upstream | this repo | |---|---|---| | download | 2,472 MB | **933 MB** | | model load | ~1.7 s | **~0.05 s** | | transcription output | reference | **identical** | Verified byte-identical transcripts against the quantize-at-load path on real speech clips (English, 5 to 45 seconds). ## Usage `parakeet_mlx.from_pretrained()` does NOT load quantized weights: it builds a full-precision model and loads the tensors into it. Build the quantized structure first, then load: ```python import json import mlx.nn as nn from huggingface_hub import hf_hub_download from parakeet_mlx.utils import from_config repo = "cosmicfusionlabs/parakeet-tdt-0.6b-v2-mlx-8bit" config = json.load(open(hf_hub_download(repo, "config.json"))) quant = config.pop("quantization") # {"bits": 8, "group_size": 64} model = from_config(config) nn.quantize(model, bits=quant["bits"], group_size=quant["group_size"]) model.load_weights(hf_hub_download(repo, "model.safetensors")) model.eval() result = model.transcribe("audio.wav") print(result.text) ``` ## Accuracy Same weights as quantizing the upstream model at load time, so accuracy is unchanged from the 8-bit path. In our own benchmark on real (non-synthetic) English speech, 8-bit Parakeet TDT 0.6B v2 measured 11.6% mean WER, between `whisper-small.en` (13.5%) and `whisper-medium.en` (8.2%), while running 2 to 4x faster than `medium.en` on an M1 Pro. 4-bit was measured too and degraded accuracy (13.3% WER), which is why this repo ships 8-bit. ## Credits and license - Original model: [nvidia/parakeet-tdt-0.6b-v2](https://h...
Source context: 12 downloads · 0 likes · Pipeline automatic-speech-recognition · Library mlx · Repo cosmicfusionlabs/parakeet-tdt-0.6b-v2-mlx-8bit
