LS EEND ONNX

These models are intended for stateful streaming inference. Each package runs one LS-EEND step at a time with explicit recurrent/cache tensors, rather than processing an entire utterance in a single call.

Included files

Each variant directory contains:

• *.onnx: the ONNX model
• *.json: metadata needed by the runtime

Variant directories:

• AMI/
• CALLHOME/
• DIHARD II/
• DIHARD III/

Variants

The metadata JSON distinguishes between:

• max_speakers: the dataset/config speaker setting from the LS-EEND infer YAML
• max_nspks: the exported model's full decode/output capacity

Frontend and runtime assumptions

All four non-quantized exports in this repo use the same frontend settings:

• sample rate: 8000 Hz
• window length: 200 samples
• hop length: 80 samples
• FFT size: 1024
• mel bins: 23
• context receptive field: 7
• subsampling: 10
• feature type: logmel23_cummn
• output frame rate: 10 Hz
• compute precision: float32

These are step-wise streaming models. A runtime must maintain and feed the recurrent state tensors between calls:

• enc_ret_kv
• enc_ret_scale
• enc_conv_cache
• dec_ret_kv
• dec_ret_scale
• top_buffer

The ONNX inputs and outputs follow the LS-EEND step export used by the reference Python and Swift runtimes.

Intended usage

Use these packages with a runtime that:

1. Resamples audio to mono 8 kHz
2. Extracts LS-EEND features with the settings above
3. Preserves model state across step calls
4. Uses ingest/decode control inputs to handle the encoder delay and final tail flush
5. Applies postprocessing such as sigmoid, thresholding, optional median filtering, and RTTM conversion outside the ONNX graph

This repository is not a drop-in replacement for generic Hugging Face transformers inference. It is meant for custom ONNX runtimes, such as:

• The Python LS-EEND ONNX runtime in the example directory

Microphone Demo

Setup the virtual environment:

# Create and activate virtual environment
python3.10 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r example/requirements.txt

To run the Python microphone inference script for the DIHARD III variant, run the following command:

python example/ls_eend_onnx_mic_gui.py --onnx-model DIHARD\ III/ls_eend_dih3_step.onnx

For the other variants, replace DIHARD\ III/ls_eend_dih3_step.onnx with the path to the desired variant:

# AMI variant
python example/ls_eend_onnx_mic_gui.py --onnx-model AMI/ls_eend_ami_step.onnx

# CALLHOME variant
python example/ls_eend_onnx_mic_gui.py --onnx-model CALLHOME/ls_eend_callhome_step.onnx

# DIHARD II variant
python example/ls_eend_onnx_mic_gui.py --onnx-model DIHARD\ II/ls_eend_dih2_step.onnx

Minimal metadata example

Each variant ships a sidecar JSON with fields like:

{
  "sample_rate": 8000,
  "win_length": 200,
  "hop_length": 80,
  "n_fft": 1024,
  "n_mels": 23,
  "context_recp": 7,
  "subsampling": 10,
  "feat_type": "logmel23_cummn",
  "frame_hz": 10.0,
  "max_speakers": 10,
  "max_nspks": 12
}

Check the variant-specific *.json file for the exact state tensor shapes and output dimensions.

Credits

• Base model: LS-EEND by Di Liang & Xiaofei Li (Westlake University). Paper: LS-EEND: Long-Form Streaming End-to-End Neural Diarization with Online Attractor Extraction (IEEE TASLP 2025). The original model is not hosted on HuggingFace; pretrained weights are available on GitHub.

Source project

These ONNX exports were produced from the LS-EEND code in the FS-EEND repository:

• GitHub: Audio-WestlakeU/FS-EEND

The export path is based on the LS-EEND ONNX step exporter and variant batch exporter in that project.

Training and evaluation context

From the source project, the reported real-world diarization error rates are:

These numbers come from the upstream LS-EEND project README and reflect the original training/evaluation setup, not a Hugging Face evaluation pipeline.

Limitations

• These models are exported for ONNX runtimes.
• They are stateful streaming step models, so they require a custom driver loop.
• They assume an 8 kHz LS-EEND frontend and will not produce matching results if you use a different spectrogram pipeline.
• Speaker identities are output as activity tracks/slots and still require downstream diarization postprocessing and speaker-slot alignment where appropriate.

License and dataset constraints

The upstream LS-EEND model/codebase used for these ONNX exports is MIT-licensed, and this repository is published as MIT accordingly.

The underlying evaluation and fine-tuning datasets still have their own access and usage terms:

• AMI
• CALLHOME
• DIHARD II
• DIHARD III

This repository redistributes ONNX exports of the LS-EEND model variants. Dataset licensing and access requirements remain governed by the original dataset providers.

Citation

If you use LS-EEND, cite the original paper:

@ARTICLE{11122273,
  author={Liang, Di and Li, Xiaofei},
  journal={IEEE Transactions on Audio, Speech and Language Processing},
  title={LS-EEND: Long-Form Streaming End-to-End Neural Diarization With Online Attractor Extraction},
  year={2025},
  volume={33},
  pages={3568-3581},
  doi={10.1109/TASLPRO.2025.3597446}
}

DIHARD II/ls_eend_dih2_step.onnx

onnx · 42.4 MB · SHA-256 5df89a22ba87…8302 · Hugging Face