Loading
Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
ONNX export of Meta's MMSFA (Massively Multilingual Speech Forced Alignment) model for CTC forced alignment.
Model source
Source description
ONNX export of Meta's MMS_FA (Massively Multilingual Speech Forced Alignment) model for CTC forced alignment.
Sources
1 sourceVerified Sep 13
Model artifacts
2 artifactsSource excerpts
2 excerpts| File | Size | Description |
|---|
mms_fa.onnx | 3.2 MB | ONNX model graph |
mms_fa.onnx.data | 1.2 GB | External weight data (FP32) |
tokenizer.json | 311 B | Token-to-ID mapping (29 tokens) |
from huggingface_hub import hf_hub_download
model_path = hf_hub_download("xycld/lyric-align-mms-fa", "mms_fa.onnx")
data_path = hf_hub_download("xycld/lyric-align-mms-fa", "mms_fa.onnx.data")
tokenizer_path = hf_hub_download("xycld/lyric-align-mms-fa", "tokenizer.json")
Note: mms_fa.onnx.data must be in the same directory as mms_fa.onnx for ONNX Runtime to load correctly. hf_hub_download handles this automatically via its cache.
| Variant | Size | Compression | Load Time | Inference | MAE | Acc @50ms | Acc @100ms | Acc @200ms | Status |
|---|---|---|---|---|---|---|---|---|---|
| FP32 (original) | 1,207 MB | 1.0x | 989ms | 424ms/line | 34.7ms | 86.2% | 97.5% | 99.4% | Available |
| FP16 | 605 MB | 2.0x | 2,335ms | 576ms/line | 34.7ms | 86.2% | 97.5% | 99.4% | Not recommended |
| UINT8 | 303 MB | 4.0x | 412ms | 262ms/line |
Benchmark: Chinese song "错位时空" (362 characters, 53 lines) on CPU.
UINT8 is the recommended variant — 75% smaller, 38% faster inference, with virtually no accuracy loss (MAE +0.2ms).
FP16 is not recommended for CPU inference (no native FP16 support, slower than FP32). INT8 (QInt8) is incompatible with some ONNX runtimes due to ConvInteger operator requirements.
This is an ONNX format conversion of Meta's MMS forced alignment model, originally distributed via torchaudio.pipelines.MMS_FA.
Original work:
Vineel Pratap, Andros Tjandra, Bowen Shi, et al. "Scaling Speech Technology to 1,000+ Languages." 2023. https://arxiv.org/abs/2305.13516
The original model weights are released by Meta under the CC-BY-NC-4.0 license. This ONNX conversion inherits the same license. Non-commercial use only.
--- license: cc-by-nc-4.0 language: zh tags: - forced-alignment - ctc - wav2vec2 - onnx - chinese - lyrics - singing pipeline_tag: audio-classification --- # MMS Forced Alignment — ONNX ONNX export of Meta's [MMS_FA (Massively Multilingual Speech Forced Alignment)](https://ai.meta.com/blog/multilingual-model-speech-recognition/) model for CTC forced alignment. ## Files | File | Size | Description | |:-----|:-----|:------------| | `mms_fa.onnx` | 3.2 MB | ONNX model graph | | `mms_fa.onnx.data` | 1.2 GB | External weight data (FP32) | | `tokenizer.json` | 311 B | Token-to-ID mapping (29 tokens) | ## Usage ```python from huggingface_hub import hf_hub_download model_path = hf_hub_download("xycld/lyric-align-mms-fa", "mms_fa.onnx") data_path = hf_hub_download("xycld/lyric-align-mms-fa", "mms_fa.onnx.data") tokenizer_path = hf_hub_download("xycld/lyric-align-mms-fa", "tokenizer.json") ``` **Note:** `mms_fa.onnx.data` must be in the same directory as `mms_fa.onnx` for ONNX Runtime to load correctly. `hf_hub_download` handles this automatically via its cache. ## Model Details - **Original model:** [MMS_FA](https://github.com/facebookresearch/fairseq/tree/main/examples/mms) by Meta Research - **Architecture:** wav2vec2-based CTC forced aligner (315M parameters) - **Precision:** FP32 - **Input:** mono 16kHz waveform - **Output:** log-probability emission matrix [num_frames, 29 labels] at 50fps (20ms/frame) - **ONNX opset:** 18 ## Quantized Variants | Variant | Size | Compression | Load Time | Inference | MAE | Acc @50ms | Acc @100ms | Acc @200ms | Status | |:--------|-----:|:-----------:|----------:|----------:|----:|----------:|-----------:|-----------:|:-------| | **FP32** (original) | 1,207 MB | 1.0x | 989ms | 424ms/line | 34.7ms | 86.2% | 97.5% | 99.4% | Available | | **FP16** | 605 MB | 2.0x | 2,335ms | 576ms/line | 34.7ms | 86.2% | 97.5% | 99.4% | Not recommended | | **UINT8** | 303 MB | 4.0x | 412ms | 262ms/line | 34.9ms | 86.2% | 97.5% | 99.4% | **Recommended** | > Benchmark: Chinese song "错位时空" (362 characters, 53 lines) on CPU. **UINT8 is the recommended variant** — 75% smaller, 38% faster inference, with virtually no accuracy loss (MAE +0.2ms). FP16 is not recommended for CPU inference (no native FP16 support, slower than FP32). INT8 (QInt8) is incompatible with some ONNX runtimes due to `ConvInteger` operator requirements. ## Att...
Source context: 0 downloads · 0 likes · Pipeline audio-classification · Repo xycld/lyric-align-mms-fa
| 34.9ms |
| 86.2% |
| 97.5% |
| 99.4% |
| Recommended |