Maya1 NVFP4

--- base_model: maya-research/maya1 base_model_relation: quantized license: apache-2.0 pipeline_tag: text-to-speech library_name: vllm tags: - text-to-speech - voice-design - voice-cloning - nvfp4 - compressed-tensors - vllm - snac language: - en --- # Maya1 — NVFP4 (compressed-tensors, for vLLM) A 4-bit **NVFP4** quantization of [`maya-research/maya1`](https://huggingface.co/maya-research/maya1) (Apache-2.0), packaged as **compressed-tensors** so it loads directly in **vLLM** with the native SM120/SM121 CUTLASS NVFP4 GEMM. Built and measured on an **NVIDIA DGX Spark (GB10, sm_121a)**; it also runs on any Blackwell + vLLM (RTX 50-series share the same warp-level FP4 path). - **~2.5 GB** weights (from 6.2 GB bf16); **~12 GB** GPU resident served. - Keeps Maya's emotion tags + natural-language voice design. ## Why this quant Single-stream TTS decode is **memory-bandwidth bound**. Quantizing the weights raises throughput on bandwidth-limited GPUs (e.g. GB10's ~273 GB/s unified memory). Measured on GB10 (decode tok/s): **bf16 28 → fp8 54 → NVFP4 ~72–75**. > **Calibration matters for a TTS model.** Generic-text calibration mis-scales the audio-token path. > This checkpoint was calibrated on **96 in-distribution Maya sequences** (real > `<description=…> text` prompts + their generated SNAC token streams), so the emotion/audio tokens > are properly represented. Quant method: `llm-compressor` `QuantizationModifier(scheme="NVFP4")`, > `lm_head` left unquantized. ## Measured (GB10, NVFP4) | Metric | Value | |---|---| | First-audio (streamed) | ~0.46 s | | Throughput | ~72–75 tok/s | | RTF | ~1.25 | | GPU resident | ~12 GB | | Sample rate | 24 kHz mono | ## How to run Maya emits **SNAC audio tokens**, not audio — you need (1) vLLM to serve the LM and (2) a thin decoder that formats the prompt, parses the SNAC tokens, and decodes them to a waveform. Both below. **1) Serve the LM with vLLM** (compressed-tensors NVFP4 is auto-detected): ```bash vllm serve <this-repo> --served-model-name maya1 \ --return-tokens-as-token-ids --max-model-len 4096 --trust-remote-code # DGX Spark / aarch64: use the vllm/vllm-openai:*-aarch64-cu130 image. ``` **2) Decode SNAC → audio** with the included wrapper (`maya_tts_server.py`), which exposes an OpenAI-compatible `/v1/audio/speech` (streaming raw PCM or wav): ```bash pip install snac fastapi uvicorn httpx soundfile n...

Source context: 6 downloads · 0 likes · Pipeline text-to-speech · Library vllm · Repo Sggin/maya1-nvfp4