Milfer Denoiser v1.0

--- language: - ru - en tags: - audio - speech - audio-restoration - pytorch - cuda pipeline_tag: audio-to-audio license: other --- # MILFER MILFER is a standalone PyTorch audio-to-audio model for speech-preserving audio restoration. It takes an input audio file, extracts SSL speech features, and reconstructs a 48 kHz waveform with the bundled neural decoder. The bundled checkpoint is `milfer_lora100h_step001000`. ## Highlights - Pure PyTorch inference, no TorchScript runtime required. - CUDA fp16 inference by default when a CUDA GPU is available. - Accepts common audio formats supported by `torchaudio`, including wav and mp3. - Emits a mono 48 kHz wav file. - Tuned to preserve more game/dialogue sound character than the base checkpoint. ## Quick Start ```bash python milfer.py input.wav output.wav ``` For CUDA fp16: ```bash python milfer.py input.mp3 output.wav --device cuda --precision fp16 ``` For repeated inference in the same Python process, compile the feature model and run one warm-up pass first: ```bash python milfer.py input.mp3 output.wav --device cuda --precision fp16 --compile-feature ``` The helper script does the same thing with the local Python environment: ```bash ./run.sh input.wav output.wav --device cuda --precision fp16 ``` ## Files ```text milfer.py run.sh weights/ decoder_state_dict.pt feature_predictor_config.json feature_predictor_state_dict.pt milfer_config.json ``` ## Requirements Tested with: - Python 3.10 - PyTorch 2.6.0 + CUDA 12.4 - torchaudio 2.6.0 - transformers - soundfile - descript-audio-codec ## Clean Input Check The table below measures how much MILFER changes already-clean clips. It is a sanity check, not a denoising benchmark. Evaluation set: `prompts_5kh`, 250 mono wav clips, 44.1 kHz, 19.0 minutes total. Higher is better for STOI, eSTOI, PESQ-WB, and MOS predictors. Lower is better for LSD and clipped samples. | Subset | Files | STOI | eSTOI | PESQ-WB | LSD 16 kHz | Clipped Samples | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | all clips | 250 | 0.9241 | 0.8719 | 2.1653 | 11.825 dB | 0.0006% | | duration >= 1 s | 232 | 0.9288 | 0.8767 | 2.1917 | 11.683 dB | 0.0005% | No-reference MOS predictors on the original and processed outputs: | Subset | Audio | UTMOS | DistillMOS | NISQA-TTS | | --- | --- | ---: | ---: | ---: | | all clips | original | 2.9998 | 3.9392 | 3.6311 | | all clips...