Parakeet Ctc 0.6b GGUF

Parakeet CTC 0.6B — GGUF (ggml-quantised)

GGUF / ggml conversions of nvidia/parakeet-ctc-0.6b for use with the crispasr CLI from CrispStrobe/CrispASR.

Parakeet CTC 0.6B is NVIDIA's 600 M-parameter English ASR model:

• English-only, lowercase output (matches the upstream training convention)
• FastConformer encoder (24 layers, d_model=1024, 8 heads) + single CTC head — single forward pass per utterance, no autoregressive joint loop
• CC-BY-4.0 licence
• Strong WERs on the standard suite: LibriSpeech-clean 1.87 %, LibriSpeech-other 3.76 %, TEDLIUM-v3 3.78 %, GigaSpeech 10.35 %, Common Voice 7.00 %

This repo provides four quantisations, all converted from the upstream .nemo checkpoint via models/convert-stt-fastconformer-ctc-to-gguf.py (the same converter used for stt_en_fastconformer_ctc_*, since parakeet-ctc-0.6b shares the FastConformer-CTC architecture) and quantised with crispasr-quantize.

Files

File	Size	Notes
parakeet-ctc-0.6b.gguf	~1.22 GB	F16, full precision
parakeet-ctc-0.6b-q8_0.gguf	~720 MB	Q8_0, near-lossless
parakeet-ctc-0.6b-q5_0.gguf	~520 MB	Q5_0
parakeet-ctc-0.6b-q4_k.gguf	~455 MB	Q4_K — recommended default

All quantisations produce the same JFK 11 s transcript.

Quick start

# 1. Build the runtime
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc) --target crispasr-cli

# 2. Run — the CLI auto-downloads Q4_K from this repo by friendly name:
./build/bin/crispasr -m parakeet-ctc-0.6b -f your-audio.wav

# Or pre-download a specific quant via huggingface_hub and point to it:
python -c "from huggingface_hub import hf_hub_download; print(hf_hub_download('cstr/parakeet-ctc-0.6b-GGUF', 'parakeet-ctc-0.6b-q8_0.gguf'))"
./build/bin/crispasr -m parakeet-ctc-0.6b-q8_0.gguf -f your-audio.wav

The crispasr CLI auto-detects the backend from filename — parakeet-ctc-*.gguf routes to fastconformer-ctc because the architecture is identical to the stt_en_fastconformer_ctc_* family. Registry key parakeet-ctc-0.6b triggers Q4_K auto-download.

Model architecture

Component	Details
Encoder	24-layer FastConformer, d=1024, 8 heads, head_dim=128, FFN=4096, conv kernel=9, attention biases ON
Subsampling	dw_striding stack, 8× temporal (50 → 12.5 fps)
CTC head	Conv1d(1024 → 1025), k=1; vocab 1024 SentencePiece + 1 blank
Audio	16 kHz mono, 80 mel bins, n_fft=512, hop=160, win=400
Parameters	~600 M

The mel filterbank and Hann window are baked into the GGUF (preprocessor.fb, preprocessor.window). BatchNorm in the convolution module is folded into the depthwise conv weights at load time.

Performance (Apple M1 Metal, JFK 11 s, q8_0)

Path	Median wallclock	RT×
crispasr ctypes Session, Metal	0.46 s	24.1×
onnx-asr (CPU EP, int8)	0.72 s	15.2×
onnx-asr (CoreML EP, int8)	1.28 s	8.6×

(Apples-to-apples on CTC-vs-CTC at the same param count vs istupakov/parakeet-ctc-0.6b-onnx. See PERFORMANCE.md for the full methodology.)

Output convention

The upstream model emits lowercase, un-punctuated English. If you need cased + punctuated output, pair with the parakeet-tdt-0.6b-v3 (cstr/parakeet-tdt-0.6b-v3-GGUF) instead, or post-process via crispasr's --punc-model (FireRedPunc / fullstop-punc).

Attribution

• Original model: nvidia/parakeet-ctc-0.6b (CC-BY-4.0). NVIDIA NeMo team.
• GGUF conversion + ggml runtime: CrispStrobe/CrispASR — FastConformer-CTC backend, see src/canary_ctc.cpp.
• Reference inference: istupakov/onnx-asr was the cross-check for the CTC head argmax / token decoding.

Related

• C++ runtime: CrispStrobe/CrispASR
• Sister TDT model: cstr/parakeet-tdt-0.6b-v3-GGUF
• Larger CTC variant: cstr/parakeet-ctc-1.1b-GGUF
• Same-size FastConformer-CTC: cstr/stt-en-fastconformer-ctc-xlarge-GGUF

License

CC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.

Provenance and EU AI Act Art. 53 note

• Upstream model: nvidia/parakeet-ctc-0.6b — published by nvidia.
• Upstream licence: cc-by-4.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.