Distilhubert Ser

--- license: apache-2.0 tags: - speech-emotion-recognition - onnx - distilhubert - arousal-valence datasets: - CREMA-D language: - en pipeline_tag: audio-classification library_name: onnxruntime --- # DistilHuBERT SER (Speech Emotion Recognition) Arousal-valence dimensional emotion recognition from speech, built on [ntu-spml/distilhubert](https://huggingface.co/ntu-spml/distilhubert). ## Model Details - **Backbone**: DistilHuBERT (23.5M params) - **Head**: Linear(768 → 256) → GELU → Dropout(0.3) → Linear(256 → 2) → Tanh - **Output**: arousal ∈ [-1, 1], valence ∈ [-1, 1] - **Input**: Raw 16kHz waveform, variable length - **Training data**: CREMA-D (7,442 samples, 91 speakers) - **Loss**: Concordance Correlation Coefficient (CCC) - **Best CCC**: arousal=0.783, valence=0.737, avg=0.760 ## Files | File | Size | Description | |------|------|-------------| | `distilhubert_ser.onnx` | 90 MB | fp32 ONNX model | | `distilhubert_ser_int8.onnx` | 48 MB | INT8 dynamic quantized (deployment) | ## Usage (ONNX Runtime) ```python import onnxruntime as ort import numpy as np session = ort.InferenceSession( "distilhubert_ser_int8.onnx", providers=["CPUExecutionProvider"] ) # audio: raw 16 kHz mono waveform, float32, peak-normalized, shape [1, N] audio = waveform.astype(np.float32)[None, :] arousal, valence = session.run(["arousal", "valence"], {"audio": audio}) # arousal, valence: float32 [1], each in [-1, 1] ``` ## Training Trained with CCC loss on CREMA-D categorical emotions mapped to dimensional centroids: - ANG → arousal=0.8, valence=-0.6 - DIS → arousal=0.3, valence=-0.7 - FEA → arousal=0.7, valence=-0.5 - HAP → arousal=0.6, valence=0.7 - NEU → arousal=0.0, valence=0.0 - SAD → arousal=-0.5, valence=-0.4 30 epochs, batch size 16, lr=1e-4, AdamW. ## Provenance and License Full provenance: `provenance.json` in this repo. Apache-2.0, matching the [ntu-spml/distilhubert](https://huggingface.co/ntu-spml/distilhubert) backbone this model fine-tunes. (This card previously said MIT; corrected 2026-07-25 to match the backbone's license and the Omote model registry.)