ModelListadosExterno

Nemotron Speech Streaming En 0.6b Coreml

nvidia-open-model-license

Fontehuggingface.co/mrinalwadhwa/nemotron-speech-streaming-en-0.6b-coreml/tree/c812c6604ec09800084fa8c38bacb6748239c48c

TipoOtherModelo basenvidia/nemotron-speech-streaming-en-0.6bVersãoRevision c812c66Cnota

Fonte do modelo

Tipo: Other
Tamanho do arquivo: 243 B–563 MB
Modelo base: nvidia/nemotron-speech-streaming-en-0.6b
Versão: vRevision c812c66
Atualizado: 27 de set. de 2026
Downloads: 5
Fonte: Hugging Face

Descrição da fonte

Nemotron Speech Streaming 0.6B - CoreML

CoreML conversion of NVIDIA's nvidia/nemotron-speech-streaming-en-0.6b for real-time streaming ASR on Apple devices.

Model Variants

Four chunk-size variants optimized for different latency/accuracy trade-offs:

Variant	Chunk Duration	Latency	Use Case
`nemotron_coreml_1120ms`	1.12s	High	Best accuracy
`nemotron_coreml_560ms`	0.56s	Medium	Balanced
`nemotron_coreml_160ms`	0.16s	Low	Real-time feedback
`nemotron_coreml_80ms`	0.08s	Ultra-low	Experimental

All variants include:

Int8 quantized encoder (~564MB, 4x smaller than float32)
Compiled .mlmodelc format (ready for deployment)

Benchmark Results (LibriSpeech test-clean)

Tested on Apple M2 with FluidAudio:

Chunk Size	WER	RTFx	Files
1120ms	1.99%	9.6x	100
560ms	2.12%	8.5x	100
160ms	~10%	3.5x	20
80ms	~60%	1.9x	20

160ms and 80ms were only tested on 20 files.

Model Overview

Property	Value
Source Model	`nvidia/nemotron-speech-streaming-en-0.6b`
Architecture	FastConformer RNNT (Streaming)
Parameters	0.6B
Sample Rate	16kHz
Mel Features	128 bins
Quantization	Int8 (encoder)

CoreML Models (per variant)

Model	Size	Function
`preprocessor.mlmodelc`	~1MB	audio → 128-dim mel spectrogram
`encoder/encoder_int8.mlmodelc`	~564MB	mel + cache → encoded + new_cache
`decoder.mlmodelc`	~28MB	token + LSTM state → decoder_out + new_state
`joint.mlmodelc`	~7MB	encoder + decoder → logits

Plus:

metadata.json - Model configuration (chunk size, mel frames, etc.)
tokenizer.json - Vocabulary (1024 tokens)

Directory Structure

nemotron-speech-streaming-en-0.6b-coreml/
├── nemotron_coreml_1120ms/      # 1.12s chunks (best accuracy)
│   ├── encoder/
│   │   └── encoder_int8.mlmodelc
│   ├── preprocessor.mlmodelc
│   ├── decoder.mlmodelc
│   ├── joint.mlmodelc
│   ├── metadata.json
│   └── tokenizer.json
├── nemotron_coreml_560ms/       # 0.56s chunks (balanced)
│   └── ...
├── nemotron_coreml_160ms/       # 0.16s chunks (low latency)
│   └── ...
└── nemotron_coreml_80ms/        # 0.08s chunks (experimental)
    └── ...

Chunk Configuration

Each variant has different mel frame counts:

Variant	chunk_mel_frames	pre_encode_cache	total_mel_frames
1120ms	112	9	121
560ms	56	9	65
160ms	16	9	25
80ms	8	9	17

Formula: chunk_ms = chunk_mel_frames × 10ms

Cache Shapes

Cache	Shape	Description
cache_channel	[1, 24, 70, 1024]	Attention context cache
cache_time	[1, 24, 1024, 8]	Convolution time cache
cache_len	[1]	Cache fill level

Usage with FluidAudio

import FluidAudio

// Load with specific chunk size
let manager = NemotronStreamingAsrManager()
let modelDir = // path to nemotron_coreml_560ms
try await manager.loadModels(modelDir: modelDir)

// Process audio
let result = try await manager.process(audioBuffer: buffer)
let transcript = try await manager.finish()

CLI Benchmark

# Install FluidAudio CLI
git clone https://github.com/FluidInference/FluidAudio
cd FluidAudio

# Run benchmark with specific chunk size
swift run -c release fluidaudiocli nemotron-benchmark --chunk 560 --max-files 100

Inference Pipeline

┌─────────────────────────────────────────────────────────────────┐
│                     STREAMING RNNT PIPELINE                      │
└─────────────────────────────────────────────────────────────────┘

1. PREPROCESSOR (per audio chunk)
   audio [1, samples] → mel [1, 128, chunk_mel_frames]

2. ENCODER (with cache)
   mel [1, 128, total_mel_frames] + cache → encoded [1, 1024, T] + new_cache
   (total_mel_frames = pre_encode_cache + chunk_mel_frames)

3. DECODER + JOINT (greedy loop per encoder frame)
   For each encoder frame:
     token → DECODER → decoder_out
     encoder_step + decoder_out → JOINT → logits
     argmax → predicted token
     if token == BLANK: next encoder frame
     else: emit token, update decoder state

Quantization Details

The encoder is quantized to int8 using CoreMLTools:

Metric	Float32	Int8
Size	~2.2GB	~564MB
Compression	1x	3.9x
WER Impact	Baseline	Negligible

Other models (preprocessor, decoder, joint) remain in float32 as they are already small.

Notes

The encoder is the largest model with 24 Conformer layers
Model uses 128 mel bins (not the typical 80)
RNNT blank token index is 1024 (vocab_size)
Decoder uses 2-layer LSTM with 640 hidden units
Pre-encode cache (9 frames = 90ms) bridges chunk boundaries

License

Apache 2.0 (following NVIDIA's original license)

Fontes

1 fonte

Hugging FacePrimárioVersão c812c6604ec09800084fa8c38bacb6748239c48c

huggingface.co/mrinalwadhwa/nemotron-speech-streaming-en-0.6b-coreml/tree/c812c6604ec09800084fa8c38bacb6748239c48c

Verificado 30 de ago.

Artefatos de modelo

36 artefatos

nemotron_coreml_1120ms/decoder_joint.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 d46ecf790a72…8ffc · Hugging Face

Baixar

nemotron_coreml_1120ms/decoder_joint.mlmodelc/coremldata.bin

bin · 514 B · SHA-256 dd0ffdc0887e…74c1 · Hugging Face

Baixar

nemotron_coreml_1120ms/decoder_joint.mlmodelc/weights/weight.bin

bin · 17,0 MB · SHA-256 96cac67a1352…5f52 · Hugging Face

Baixar

nemotron_coreml_1120ms/decoder.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 1e71eacbdeef…b6f6 · Hugging Face

Baixar

nemotron_coreml_1120ms/decoder.mlmodelc/coremldata.bin

bin · 493 B · SHA-256 3fd85d699062…64a8 · Hugging Face

Baixar

nemotron_coreml_1120ms/decoder.mlmodelc/weights/weight.bin

bin · 13,8 MB · SHA-256 d9758e49f9b2…7818 · Hugging Face

Baixar

nemotron_coreml_1120ms/encoder/encoder_int8.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 bfa033f77995…e04a · Hugging Face

Baixar

nemotron_coreml_1120ms/encoder/encoder_int8.mlmodelc/coremldata.bin

bin · 608 B · SHA-256 4ddbee0baaa8…ad01 · Hugging Face

Baixar

nemotron_coreml_1120ms/encoder/encoder_int8.mlmodelc/weights/weight.bin

bin · 562 MB · SHA-256 409df1105de0…f715 · Hugging Face

Baixar

nemotron_coreml_1120ms/joint.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 9191ce654777…6076 · Hugging Face

Baixar

nemotron_coreml_1120ms/joint.mlmodelc/coremldata.bin

bin · 401 B · SHA-256 70798ae50692…ea0d · Hugging Face

Baixar

nemotron_coreml_1120ms/joint.mlmodelc/weights/weight.bin

bin · 3,29 MB · SHA-256 d1d45514a91d…06d5 · Hugging Face

Baixar

nemotron_coreml_2240ms/decoder_joint.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 d46ecf790a72…8ffc · Hugging Face

Baixar

nemotron_coreml_2240ms/decoder_joint.mlmodelc/coremldata.bin

bin · 514 B · SHA-256 0161ed3e8117…25c7 · Hugging Face

Baixar

nemotron_coreml_2240ms/decoder_joint.mlmodelc/weights/weight.bin

bin · 17,0 MB · SHA-256 96cac67a1352…5f52 · Hugging Face

Baixar

nemotron_coreml_2240ms/decoder.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 1e71eacbdeef…b6f6 · Hugging Face

Baixar

nemotron_coreml_2240ms/decoder.mlmodelc/coremldata.bin

bin · 493 B · SHA-256 403aa3032170…88cb · Hugging Face

Baixar

nemotron_coreml_2240ms/decoder.mlmodelc/weights/weight.bin

bin · 13,8 MB · SHA-256 d9758e49f9b2…7818 · Hugging Face

Baixar

nemotron_coreml_2240ms/encoder/encoder_int8.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 4766055cb558…f223 · Hugging Face

Baixar

nemotron_coreml_2240ms/encoder/encoder_int8.mlmodelc/coremldata.bin

bin · 609 B · SHA-256 3f863b74a80d…6349 · Hugging Face

Baixar

nemotron_coreml_2240ms/encoder/encoder_int8.mlmodelc/weights/weight.bin

bin · 563 MB · SHA-256 0c67d14d95f9…a7eb · Hugging Face

Baixar

nemotron_coreml_2240ms/joint.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 9191ce654777…6076 · Hugging Face

Baixar

nemotron_coreml_2240ms/joint.mlmodelc/coremldata.bin

bin · 401 B · SHA-256 70798ae50692…ea0d · Hugging Face

Baixar

nemotron_coreml_2240ms/joint.mlmodelc/weights/weight.bin

bin · 3,29 MB · SHA-256 d1d45514a91d…06d5 · Hugging Face

Baixar

nemotron_coreml_560ms/decoder_joint.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 d46ecf790a72…8ffc · Hugging Face

Baixar

nemotron_coreml_560ms/decoder_joint.mlmodelc/coremldata.bin

bin · 514 B · SHA-256 d9a7f7cfece9…2172 · Hugging Face

Baixar

nemotron_coreml_560ms/decoder_joint.mlmodelc/weights/weight.bin

bin · 17,0 MB · SHA-256 96cac67a1352…5f52 · Hugging Face

Baixar

nemotron_coreml_560ms/decoder.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 bca0e343fa0d…29b8 · Hugging Face

Baixar

nemotron_coreml_560ms/decoder.mlmodelc/coremldata.bin

bin · 493 B · SHA-256 4d0549ac72a3…4f4e · Hugging Face

Baixar

nemotron_coreml_560ms/decoder.mlmodelc/weights/weight.bin

bin · 13,8 MB · SHA-256 d9758e49f9b2…7818 · Hugging Face

Baixar

nemotron_coreml_560ms/encoder/encoder_int8.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 ef51dbab5c6c…6daa · Hugging Face

Baixar

nemotron_coreml_560ms/encoder/encoder_int8.mlmodelc/coremldata.bin

bin · 608 B · SHA-256 864a357ae2eb…c404 · Hugging Face

Baixar

nemotron_coreml_560ms/encoder/encoder_int8.mlmodelc/weights/weight.bin

bin · 562 MB · SHA-256 8ced31411f79…3a40 · Hugging Face

Baixar

nemotron_coreml_560ms/joint.mlmodelc/analytics/coremldata.bin

bin · 243 B · SHA-256 478804308534…b103 · Hugging Face

Baixar

nemotron_coreml_560ms/joint.mlmodelc/coremldata.bin

bin · 401 B · SHA-256 46363626fd89…abb2 · Hugging Face

Baixar

nemotron_coreml_560ms/joint.mlmodelc/weights/weight.bin

bin · 3,29 MB · SHA-256 d1d45514a91d…06d5 · Hugging Face

Baixar

Trechos de fonte

2 trechos

Readme30 de ago.huggingface.co/mrinalwadhwa/nemotron-speech-streaming-en-0.6b-coreml

--- license: other license_name: nvidia-open-model-license license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ library_name: coreml base_model: nvidia/nemotron-speech-streaming-en-0.6b tags: - speech-recognition - automatic-speech-recognition - streaming-asr - coreml - apple - ios - macos - FastConformer - RNNT - Parakeet - ASR pipeline_tag: automatic-speech-recognition --- # Nemotron Speech Streaming 0.6B - CoreML CoreML conversion of NVIDIA's `nvidia/nemotron-speech-streaming-en-0.6b` for real-time streaming ASR on Apple devices. ## Model Variants Four chunk-size variants optimized for different latency/accuracy trade-offs: | Variant | Chunk Duration | Latency | Use Case | |---------|---------------|---------|----------| | `nemotron_coreml_1120ms` | 1.12s | High | Best accuracy | | `nemotron_coreml_560ms` | 0.56s | Medium | Balanced | | `nemotron_coreml_160ms` | 0.16s | Low | Real-time feedback | | `nemotron_coreml_80ms` | 0.08s | Ultra-low | Experimental | All variants include: - **Int8 quantized encoder** (~564MB, 4x smaller than float32) - **Compiled .mlmodelc format** (ready for deployment) ## Benchmark Results (LibriSpeech test-clean) Tested on Apple M2 with [FluidAudio](https://github.com/FluidInference/FluidAudio): | Chunk Size | WER | RTFx | Files | |------------|-----|------|-------| | 1120ms | 1.99% | 9.6x | 100 | | 560ms | 2.12% | 8.5x | 100 | | 160ms | ~10% | 3.5x | 20 | | 80ms | ~60% | 1.9x | 20 | 160ms and 80ms were only tested on 20 files. ## Model Overview | Property | Value | |----------|-------| | Source Model | `nvidia/nemotron-speech-streaming-en-0.6b` | | Architecture | FastConformer RNNT (Streaming) | | Parameters | 0.6B | | Sample Rate | 16kHz | | Mel Features | 128 bins | | Quantization | Int8 (encoder) | ## CoreML Models (per variant) | Model | Size | Function | |-------|------|----------| | `preprocessor.mlmodelc` | ~1MB | audio → 128-dim mel spectrogram | | `encoder/encoder_int8.mlmodelc` | ~564MB | mel + cache → encoded + new_cache | | `decoder.mlmodelc` | ~28MB | token + LSTM state → decoder_out + new_state | | `joint.mlmodelc` | ~7MB | encoder + decoder → logits | Plus: - `metadata.json` - Model configuration (chunk size, mel frames, etc.) - `tokenizer.json` - Vocabulary (1024 tokens) ## Directory Structure ``` nemotron-speech-streaming-en-0.6b-core...

Note30 de ago.huggingface.co/mrinalwadhwa/nemotron-speech-streaming-en-0.6b-coreml

Source context: 5 downloads · 0 likes · Pipeline automatic-speech-recognition · Library coreml · Repo mrinalwadhwa/nemotron-speech-streaming-en-0.6b-coreml