Mpse Smoe Speech Enhancement
Speech enhancement model handling both Narrowband (NB, 8kHz) and Wideband (WB, 16kHz) audio using Supervised Mixture of Experts.
Modellquelle
- Art
- Other
- Dateigröße
- 8,72 MB
- Version
- vRevision 7b1b3e8
- Aktualisiert
- 31. Okt. 2026
- Downloads
- 22
- Quelle
- Hugging Face
Quellenbeschreibung
MP-SENet + Supervised Mixture of Experts (S-MoE) for Dual-Bandwidth Speech Enhancement
Speech enhancement model handling both Narrowband (NB, 8kHz) and Wideband (WB, 16kHz) audio using Supervised Mixture of Experts.
Based on:
- MP-SENet — STFT-domain magnitude-phase speech enhancement backbone
- S-MoE — Supervised MoE with deterministic hard gating by bandwidth metadata
Pre-trained Baseline
The official pre-trained MP-SENet baseline (from yxlu-0102/MP-SENet) is included at pretrained/g_best_vb.pt:
- Trained on VoiceBank+DEMAND, 16kHz, 100 epochs
- 2.263M params, 247 state dict keys
- No need to train baseline from scratch — S-MoE training initializes directly from this checkpoint
Training (S-MoE only)
pip install -r requirements-gpu.txt
# S-MoE training (downloads pre-trained baseline automatically)
SMOE_EPOCHS=60 BATCH_SIZE=2 GRAD_ACCUM_STEPS=2 python train.py
The script automatically:
- Downloads VoiceBank+DEMAND (~5GB)
- Pre-extracts WB/NB data into separate folders (no on-the-fly resampling)
- Downloads official pre-trained MP-SENet baseline from this repo
- Initializes S-MoE experts from baseline (shared → both WB+NB experts)
- Trains S-MoE on joint WB+NB data for 60 epochs
- Pushes trained model to HF Hub
Data Pipeline
All NB/WB data is pre-extracted to separate folders before training:
| Folder | Type | Processing | bandwidth_id |
|---|---|---|---|
original_wb/ | WB | Original 16kHz (symlinks) | 0 |
simple_nb/ | NB | 16k→8k→16k resampled | 1 |
codec_nb/ | NB | G.711 A-law/mu-law (round-robin) | 1 |
Architecture
| Component | Baseline | S-MoE |
|---|---|---|
| Total params | 2.263M | 3.587M |
| Active params (inference) | 2.263M | 2.263M (same!) |
| FFN type | Single GRU-FFN | 2× GRU-FFN experts, hard gated by bandwidth_id |
| Attention | Shared | Shared (unchanged) |
Environment Variables
| Variable | Default | Description |
|---|---|---|
SMOE_EPOCHS | 60 | Epochs for S-MoE training |
BATCH_SIZE | 2 | GPU micro-batch size |
GRAD_ACCUM_STEPS | 2 | Gradient accumulation (effective BS = BATCH_SIZE × GRAD_ACCUM_STEPS) |
PESQ_EVERY_N | 10 | Compute PESQ for discriminator every N steps (expensive) |
LR | 5e-4 | Learning rate |
EVAL_INTERVAL | 5 | Evaluate every N epochs |
BASELINE_REPO | AdityaRaikar/mpse-smoe-speech-enhancement | HF repo with pre-trained baseline |
BASELINE_FILE | pretrained/g_best_vb.pt | Checkpoint filename in repo |
Files
| File | Description |
|---|---|
train.py | Combined data prep + S-MoE training (self-contained) |
pretrained/g_best_vb.pt | Official pre-trained MP-SENet baseline (VoiceBank+DEMAND) |
smoe_models.py | Model definitions with detailed docstrings |
data_preparation.py | Standalone NB/WB codec pipeline |
config.json | Default hyperparameters |
References
@article{lu2023mpse,
title={MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra},
author={Lu, Ye-Xin and Ai, Yang and Ling, Zhen-Hua},
journal={arXiv preprint arXiv:2305.13686},
year={2023}
}
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 31. Okt.
Download files
Repository files
File inventory only; completeness is unverified.
Modellartefakte
1 Artefakt
Modellartefakte
1 ArtefaktQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- tags: - speech-enhancement - audio-to-audio - MP-SENet - supervised-mixture-of-experts - S-MoE - dual-bandwidth - narrowband - wideband license: mit --- # MP-SENet + Supervised Mixture of Experts (S-MoE) for Dual-Bandwidth Speech Enhancement **Speech enhancement model handling both Narrowband (NB, 8kHz) and Wideband (WB, 16kHz) audio** using Supervised Mixture of Experts. Based on: - **[MP-SENet](https://arxiv.org/abs/2305.13686)** — STFT-domain magnitude-phase speech enhancement backbone - **[S-MoE](https://arxiv.org/abs/2508.10009)** — Supervised MoE with deterministic hard gating by bandwidth metadata ## Pre-trained Baseline The official pre-trained MP-SENet baseline (from [yxlu-0102/MP-SENet](https://github.com/yxlu-0102/MP-SENet)) is included at `pretrained/g_best_vb.pt`: - Trained on VoiceBank+DEMAND, 16kHz, 100 epochs - 2.263M params, 247 state dict keys - **No need to train baseline from scratch** — S-MoE training initializes directly from this checkpoint ## Training (S-MoE only) ```bash pip install -r requirements-gpu.txt # S-MoE training (downloads pre-trained baseline automatically) SMOE_EPOCHS=60 BATCH_SIZE=2 GRAD_ACCUM_STEPS=2 python train.py ``` The script automatically: 1. Downloads VoiceBank+DEMAND (~5GB) 2. Pre-extracts WB/NB data into separate folders (no on-the-fly resampling) 3. Downloads official pre-trained MP-SENet baseline from this repo 4. Initializes S-MoE experts from baseline (shared → both WB+NB experts) 5. Trains S-MoE on joint WB+NB data for 60 epochs 6. Pushes trained model to HF Hub ## Data Pipeline All NB/WB data is **pre-extracted to separate folders** before training: | Folder | Type | Processing | bandwidth_id | |---|---|---|---| | `original_wb/` | WB | Original 16kHz (symlinks) | 0 | | `simple_nb/` | NB | 16k→8k→16k resampled | 1 | | `codec_nb/` | NB | G.711 A-law/mu-law (round-robin) | 1 | ## Architecture | Component | Baseline | S-MoE | |---|---|---| | Total params | 2.263M | 3.587M | | Active params (inference) | 2.263M | 2.263M (same!) | | FFN type | Single GRU-FFN | 2× GRU-FFN experts, hard gated by bandwidth_id | | Attention | Shared | Shared (unchanged) | ## Environment Variables | Variable | Default | Description | |---|---|---| | `SMOE_EPOCHS` | `60` | Epochs for S-MoE training | | `BATCH_SIZE` | `2` | GPU micro-batch size | | `GRAD_ACCUM_STEPS` | `2` | Gradient accumulation (effec...
Source context: 22 downloads · 0 likes · Pipeline audio-to-audio · Repo AdityaRaikar/mpse-smoe-speech-enhancement
Source context: 10 downloads · 0 likes · Pipeline audio-to-audio · Repo AdityaRaikar/mpse-smoe-speech-enhancement
