VoiceRestore

--- license: mit pipeline_tag: audio-to-audio library_name: transformers --- # VoiceRestore: Flow-Matching Transformers for Speech Recording Quality Restoration VoiceRestore is a cutting-edge speech restoration model designed to significantly enhance the quality of degraded voice recordings. Leveraging flow-matching transformers, this model excels at addressing a wide range of audio imperfections commonly found in speech, including background noise, reverberation, distortion, and signal loss. It is based on this [repo](https://github.com/skirdey/voicerestore) & demo of audio restorations: [VoiceRestore](https://sparkling-rabanadas-3082be.netlify.app/) ## Usage - using Transformers 🤗 ``` bash !git lfs install !git clone https://huggingface.co/jadechoghari/VoiceRestore %cd VoiceRestore !pip install -r requirements.txt ``` ``` python from transformers import AutoModel # path to the model folder (on colab it's as follows) checkpoint_path = "/content/VoiceRestore" model = AutoModel.from_pretrained(checkpoint_path, trust_remote_code=True) model("test_input.wav", "test_output.wav") #add short=False if audio is > 10 seconds model("long.mp3", "long_output.mp3", short=False) ``` ## Example ### Degraded Input: ### Degraded Input Audio <audio controls> <source src="https://huggingface.co/jadechoghari/VoiceRestore/resolve/main/test_input.wav" type="audio/mpeg"> Your browser does not support the audio element. </audio> --- ### Restored (steps=32, cfg=1.0): <audio controls> <source src="https://huggingface.co/jadechoghari/VoiceRestore/resolve/main/test_output.wav" type="audio/mpeg"> Your browser does not support the audio element. </audio> Restored audio - 16 steps, strength 0.5: --- ## Key Features - **Universal Restoration**: The model can handle any level and type of voice recording degradation. Pure magic. - **Easy to Use**: Simple interface for processing degraded audio files. - **Pretrained Model**: Includes a 301 million parameter transformer model with pre-trained weights. (Model is still in the process of training, there will be further checkpoint updates) --- ## Model Details - **Architecture**: Flow-matching transformer - **Parameters**: 300M+ parameters - **Input**: Degraded speech audio (various formats supported) - **Output**: Restored speech ## Limitations and Future Work - Current model is optimized for speech; may not perform optim...