CleanMel

--- language: - en - cn license: bigscience-openrail-m pipeline_tag: audio-to-audio --- # CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR The CleanMel model was presented in the paper [CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR](https://huggingface.co/papers/2502.20040). CleanMel is a single-channel Mel-spectrogram denoising and dereverberation network designed to improve both speech quality and automatic speech recognition (ASR) performance. It takes noisy and reverberant microphone recordings as input and predicts the corresponding clean Mel-spectrogram. This enhanced Mel-spectrogram can then be either transformed to a speech waveform with a neural vocoder or directly used for ASR. The proposed network employs interleaved cross-band and narrow-band processing in the Mel-frequency domain, which allows it to learn full-band spectral patterns and narrow-band properties of signals, respectively. A key advantage of Mel-spectrogram enhancement, compared to linear-frequency domain or time-domain speech enhancement, is that Mel-frequency presents speech in a more compact way, making it easier to learn. This compactness benefits both speech quality and ASR. Experimental results on five English and one Chinese datasets demonstrate significant improvements. * 📚 [Paper](https://huggingface.co/papers/2502.20040) * 🌐 [Project Page](https://audio.westlake.edu.cn/Research/CleanMel.html) * 💻 [GitHub Repository](https://github.com/Audio-WestlakeU/CleanMel) * 🚀 [Hugging Face Demo](https://huggingface.co/spaces/SaoYear/CleanMel) ## Overview 🚀 <p align="center"><img src="https://github.com/Audio-WestlakeU/CleanMel/raw/main/src/imgs/cleanmel_arch.png" alt="CleanMel Architecture" width="60%"/></p> **CleanMel** enhances logMel spectrograms for improved speech quality and ASR performance. Outputs are compatible with: - 🎙️ Vocoders for enhanced waveforms - 🤖 ASR systems for transcription ## Quick Start ⚡ ### Environment Setup ```bash conda create -n CleanMel python=3.10.14 conda activate CleanMel pip install -r requirements.txt ``` ### Inference Pretrained models can be downloaded manually from the [WestlakeAudioLab/CleanMel](https://huggingface.co/WestlakeAudioLab/CleanMel) repository, or automatically with the help of the `huggingface-hub` package. ```bash # Inference with pretrained models...

Source context: 0 downloads · 1 likes · Pipeline audio-to-audio · Repo WestlakeAudioLab/CleanMel