Stftvae

--- license: mit library_name: stftvae pipeline_tag: audio-to-audio tags: - audio - codec - vae - stft - neural-audio --- # STFT-VAE Continuous (VAE) neural audio codec built on the STFT. Audio is transformed to a complex spectrogram, a transformer encoder downsamples it to a low-rate continuous latent, and a transformer decoder + ISTFT reconstruct the waveform — phase is carried by the STFT/ISTFT rather than a learned vocoder. - **Sample rate:** 24 kHz - **Latent:** 3.125 Hz frame rate, 128-dim continuous (no quantization) - **Params:** ~116M - **Files:** `config.json` (architecture) + `model.safetensors` (fp32 weights) ## Usage ```bash pip install "stftvae[cli]" ``` ```python import torch from stftvae import STFTVAE vae = STFTVAE.from_pretrained("fluxions/stftvae", device="cuda") audio = torch.randn(1, 24000) # (1, T) mono @ 24 kHz recons = vae.reconstruct(audio) # (1, 1, T) # or encode / decode separately latent, length = vae.encode(audio) # (1, 128, T_latent) recons = vae.decode(latent, length=length) ``` Command line: ```bash python -m stftvae input.wav output.wav --model fluxions/stftvae ``` The encoder returns the deterministic posterior mean (no sampling noise at inference). Code: https://github.com/fluxions-ai/stftvae ## License MIT

Source context: 16 downloads · 0 likes · Pipeline audio-to-audio · Library stftvae · Repo fluxions/stftvae

Source context: 72 downloads · 0 likes · Pipeline audio-to-audio · Library stftvae · Repo fluxions/stftvae