Encoder Onnx
stability-ai-community-license
Fonte do modelo
- Tipo
- Other
- Tamanho do arquivo
- 775 KB–34,0 MB
- Modelo base
- stabilityai/stable-audio-3-small-music
- Versão
- vEncoder q4
- Atualizado
- 22 de out. de 2026
- Fonte
- Hugging Face
Descrição da fonte
stable-audio-3-small-music — ONNX audio encoder (int4)
The encoder half of the stabilityai/stable-audio-3-small-music
autoencoder, exported to ONNX so audio can be turned into latents in the browser
via onnxruntime-web.
It is the missing counterpart to the decoder in
lsb/stable-audio-3-small-music-onnx:
the decoder maps latents → audio, this maps audio → latents. Together they enable
audio-to-audio (variation), inpainting, and continuation on top of that text-to-music bundle.
The encode path is patchify (patch_size=256) → SAME encoder (taae_v2) → softnorm bottleneck.
Linear/MatMul weights are quantized to int4 MatMulNBits (block_size=32, symmetric);
convolutions and norms stay fp32. Single file, no external data.
I/O
input audio float32 (1, 2, N) stereo, 44.1 kHz, N a multiple of 8192
output latents float32 (1, 256, N/4096) same latent space as the decoder's input
N must be a multiple of 8192 samples (the model's audio_align, so the latent length
t_lat = N/4096 is even). Pad shorter clips with zeros; the latent is laid out in time, so
you can trim trailing latent frames that correspond to the padding.
Browser usage (sketch)
import * as ort from "onnxruntime-web/wasm";
ort.env.wasm.numThreads = 1; ort.env.wasm.simd = true;
const base = "https://huggingface.co/bgkb/onnx-encoder/resolve/main";
const buf = new Uint8Array(await (await fetch(`${base}/encoder_q4.onnx`)).arrayBuffer());
const sess = await ort.InferenceSession.create(buf, { executionProviders: ["wasm"] });
// audio: Float32Array of interleaved-by-channel data [L(0..N-1), R(0..N-1)], N % 8192 === 0
const latents = (await sess.run({ audio: new ort.Tensor("float32", audio, [1, 2, N]) })).latents;
// → feed latents to the diffusion (variation) or straight to the decoder (round-trip)
Quality
Measured as the full round-trip ONNX encoder → the q4 browser decoder on a real clip:
| precision | size | reconstruction SNR |
|---|---|---|
| fp32 | 215 MB | 10.2 dB |
| int4 (this file) | 36 MB | 8.2 dB |
The int4 export is bit-faithful to the fp32 graph except for the quantized weights (latent correlation 0.978 vs fp32). For variation/img2img the encoder latent is re-noised and refined by the diffusion sampler, so the int4 precision loss is not audible in practice.
License
Inherits the Stability AI Community License from the upstream weights; the bundled T5Gemma text encoder (used elsewhere in the pipeline) additionally falls under Google's Gemma Terms of Use. This repo contains only the audio autoencoder's encoder.
Fontes
1 fonte
Fontes
1 fonteVerificado 7 de ago.
Artefatos de modelo
3 artefatos
Artefatos de modelo
3 artefatosTrechos de fonte
2 trechos
Trechos de fonte
2 trechos--- license: other license_name: stability-ai-community-license license_link: https://huggingface.co/stabilityai/stable-audio-3-small-music/blob/main/LICENSE.md tags: - audio - audio-to-audio - stable-audio - onnx - q4 - matmulnbits - onnxruntime-web library_name: onnxruntime base_model: stabilityai/stable-audio-3-small-music --- # stable-audio-3-small-music — ONNX audio encoder (int4) The **encoder** half of the [`stabilityai/stable-audio-3-small-music`](https://huggingface.co/stabilityai/stable-audio-3-small-music) autoencoder, exported to ONNX so audio can be turned into latents **in the browser** via [`onnxruntime-web`](https://www.npmjs.com/package/onnxruntime-web). It is the missing counterpart to the decoder in [`lsb/stable-audio-3-small-music-onnx`](https://huggingface.co/lsb/stable-audio-3-small-music-onnx): the decoder maps latents → audio, this maps **audio → latents**. Together they enable audio-to-audio (variation), inpainting, and continuation on top of that text-to-music bundle. The encode path is `patchify (patch_size=256) → SAME encoder (taae_v2) → softnorm bottleneck`. Linear/MatMul weights are quantized to **int4 MatMulNBits** (`block_size=32`, symmetric); convolutions and norms stay fp32. Single file, no external data. ## I/O ``` input audio float32 (1, 2, N) stereo, 44.1 kHz, N a multiple of 8192 output latents float32 (1, 256, N/4096) same latent space as the decoder's input ``` `N` **must be a multiple of 8192 samples** (the model's `audio_align`, so the latent length `t_lat = N/4096` is even). Pad shorter clips with zeros; the latent is laid out in time, so you can trim trailing latent frames that correspond to the padding. ## Browser usage (sketch) ```js import * as ort from "onnxruntime-web/wasm"; ort.env.wasm.numThreads = 1; ort.env.wasm.simd = true; const base = "https://huggingface.co/bgkb/onnx-encoder/resolve/main"; const buf = new Uint8Array(await (await fetch(`${base}/encoder_q4.onnx`)).arrayBuffer()); const sess = await ort.InferenceSession.create(buf, { executionProviders: ["wasm"] }); // audio: Float32Array of interleaved-by-channel data [L(0..N-1), R(0..N-1)], N % 8192 === 0 const latents = (await sess.run({ audio: new ort.Tensor("float32", audio, [1, 2, N]) })).latents; // → feed latents to the diffusion (variation) or straight to the decoder (round-trip) ``` ## Quality Measured as the full round-tr...
Source context: 0 downloads · 0 likes · Pipeline audio-to-audio · Library onnxruntime · Repo bgkb/encoder-onnx
