ModelPrésentéExterne

Mpse Smoe Speech Enhancement

Speech enhancement model handling both Narrowband (NB, 8kHz) and Wideband (WB, 16kHz) audio using Supervised Mixture of Experts.

Sourcehuggingface.co/AdityaRaikar/mpse-smoe-speech-enhancement/tree/7b1b3e8c504006c581c6f042f2b3aaf7dd9b95ee

TypeOtherVersionRevision 7b1b3e8BClasse

Source du modèle

Type: Other
Taille du fichier: 8,72 MB
Version: vRevision 7b1b3e8
Mis à jour: 31 oct. 2026
Téléchargements: 22
Source: Hugging Face

Description de la source

MP-SENet + Supervised Mixture of Experts (S-MoE) for Dual-Bandwidth Speech Enhancement

Speech enhancement model handling both Narrowband (NB, 8kHz) and Wideband (WB, 16kHz) audio using Supervised Mixture of Experts.

Based on:

MP-SENet — STFT-domain magnitude-phase speech enhancement backbone
S-MoE — Supervised MoE with deterministic hard gating by bandwidth metadata

Pre-trained Baseline

The official pre-trained MP-SENet baseline (from yxlu-0102/MP-SENet) is included at pretrained/g_best_vb.pt:

Trained on VoiceBank+DEMAND, 16kHz, 100 epochs
2.263M params, 247 state dict keys
No need to train baseline from scratch — S-MoE training initializes directly from this checkpoint

Training (S-MoE only)

pip install -r requirements-gpu.txt

# S-MoE training (downloads pre-trained baseline automatically)
SMOE_EPOCHS=60 BATCH_SIZE=2 GRAD_ACCUM_STEPS=2 python train.py

The script automatically:

Downloads VoiceBank+DEMAND (~5GB)
Pre-extracts WB/NB data into separate folders (no on-the-fly resampling)
Downloads official pre-trained MP-SENet baseline from this repo
Initializes S-MoE experts from baseline (shared → both WB+NB experts)
Trains S-MoE on joint WB+NB data for 60 epochs
Pushes trained model to HF Hub

Data Pipeline

All NB/WB data is pre-extracted to separate folders before training:

Folder	Type	Processing	bandwidth_id
`original_wb/`	WB	Original 16kHz (symlinks)	0
`simple_nb/`	NB	16k→8k→16k resampled	1
`codec_nb/`	NB	G.711 A-law/mu-law (round-robin)	1

Architecture

Component	Baseline	S-MoE
Total params	2.263M	3.587M
Active params (inference)	2.263M	2.263M (same!)
FFN type	Single GRU-FFN	2× GRU-FFN experts, hard gated by bandwidth_id
Attention	Shared	Shared (unchanged)

Environment Variables

Variable	Default	Description
`SMOE_EPOCHS`	`60`	Epochs for S-MoE training
`BATCH_SIZE`	`2`	GPU micro-batch size
`GRAD_ACCUM_STEPS`	`2`	Gradient accumulation (effective BS = BATCH_SIZE × GRAD_ACCUM_STEPS)
`PESQ_EVERY_N`	`10`	Compute PESQ for discriminator every N steps (expensive)
`LR`	`5e-4`	Learning rate
`EVAL_INTERVAL`	`5`	Evaluate every N epochs
`BASELINE_REPO`	`AdityaRaikar/mpse-smoe-speech-enhancement`	HF repo with pre-trained baseline
`BASELINE_FILE`	`pretrained/g_best_vb.pt`	Checkpoint filename in repo

Files

File	Description
`train.py`	Combined data prep + S-MoE training (self-contained)
`pretrained/g_best_vb.pt`	Official pre-trained MP-SENet baseline (VoiceBank+DEMAND)
`smoe_models.py`	Model definitions with detailed docstrings
`data_preparation.py`	Standalone NB/WB codec pipeline
`config.json`	Default hyperparameters

References

@article{lu2023mpse,
  title={MP-SENet: A Speech Enhancement Model with Parallel Denoising of Magnitude and Phase Spectra},
  author={Lu, Ye-Xin and Ai, Yang and Ling, Zhen-Hua},
  journal={arXiv preprint arXiv:2305.13686},
  year={2023}
}

Sources

1 source

Hugging FacePrincipalVersion 7b1b3e8c504006c581c6f042f2b3aaf7dd9b95ee

huggingface.co/AdityaRaikar/mpse-smoe-speech-enhancement/tree/7b1b3e8c504006c581c6f042f2b3aaf7dd9b95ee

Vérifié 31 oct.

Download files

Repository files

File inventory only; completeness is unverified.

Artefacts du modèle

1 artefact

pretrained/g_best_vb.pt

pt · 8,72 MB · SHA-256 aedfb1aa5491…f4f9 · Hugging Face

Download file

Extraits de sources

3 extraits

Readme14 aoûthuggingface.co/AdityaRaikar/mpse-smoe-speech-enhancement

--- tags: - speech-enhancement - audio-to-audio - MP-SENet - supervised-mixture-of-experts - S-MoE - dual-bandwidth - narrowband - wideband license: mit --- # MP-SENet + Supervised Mixture of Experts (S-MoE) for Dual-Bandwidth Speech Enhancement **Speech enhancement model handling both Narrowband (NB, 8kHz) and Wideband (WB, 16kHz) audio** using Supervised Mixture of Experts. Based on: - **[MP-SENet](https://arxiv.org/abs/2305.13686)** — STFT-domain magnitude-phase speech enhancement backbone - **[S-MoE](https://arxiv.org/abs/2508.10009)** — Supervised MoE with deterministic hard gating by bandwidth metadata ## Pre-trained Baseline The official pre-trained MP-SENet baseline (from [yxlu-0102/MP-SENet](https://github.com/yxlu-0102/MP-SENet)) is included at `pretrained/g_best_vb.pt`: - Trained on VoiceBank+DEMAND, 16kHz, 100 epochs - 2.263M params, 247 state dict keys - **No need to train baseline from scratch** — S-MoE training initializes directly from this checkpoint ## Training (S-MoE only) ```bash pip install -r requirements-gpu.txt # S-MoE training (downloads pre-trained baseline automatically) SMOE_EPOCHS=60 BATCH_SIZE=2 GRAD_ACCUM_STEPS=2 python train.py ``` The script automatically: 1. Downloads VoiceBank+DEMAND (~5GB) 2. Pre-extracts WB/NB data into separate folders (no on-the-fly resampling) 3. Downloads official pre-trained MP-SENet baseline from this repo 4. Initializes S-MoE experts from baseline (shared → both WB+NB experts) 5. Trains S-MoE on joint WB+NB data for 60 epochs 6. Pushes trained model to HF Hub ## Data Pipeline All NB/WB data is **pre-extracted to separate folders** before training: | Folder | Type | Processing | bandwidth_id | |---|---|---|---| | `original_wb/` | WB | Original 16kHz (symlinks) | 0 | | `simple_nb/` | NB | 16k→8k→16k resampled | 1 | | `codec_nb/` | NB | G.711 A-law/mu-law (round-robin) | 1 | ## Architecture | Component | Baseline | S-MoE | |---|---|---| | Total params | 2.263M | 3.587M | | Active params (inference) | 2.263M | 2.263M (same!) | | FFN type | Single GRU-FFN | 2× GRU-FFN experts, hard gated by bandwidth_id | | Attention | Shared | Shared (unchanged) | ## Environment Variables | Variable | Default | Description | |---|---|---| | `SMOE_EPOCHS` | `60` | Epochs for S-MoE training | | `BATCH_SIZE` | `2` | GPU micro-batch size | | `GRAD_ACCUM_STEPS` | `2` | Gradient accumulation (effec...

Note31 oct.huggingface.co/AdityaRaikar/mpse-smoe-speech-enhancement

Source context: 22 downloads · 0 likes · Pipeline audio-to-audio · Repo AdityaRaikar/mpse-smoe-speech-enhancement

Note14 aoûthuggingface.co/AdityaRaikar/mpse-smoe-speech-enhancement

Source context: 10 downloads · 0 likes · Pipeline audio-to-audio · Repo AdityaRaikar/mpse-smoe-speech-enhancement