Vits Melo Tts Zh En 8k

--- license: mit language: - zh - en pipeline_tag: text-to-speech tags: - vits - melo-tts - sherpa-onnx - onnx - vocoder-distillation - 8khz - telephony - jetson-nano - edge base_model: myshell-ai/MeloTTS-Chinese --- # vits-melo-tts-zh_en-8k — MeloTTS zh_en with a distilled 8 kHz vocoder Drop-in replacement for the [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) `vits-melo-tts-zh_en` model directory, with the 44.1 kHz HiFi-GAN decoder replaced by a **distilled 8 kHz Vocos-style vocoder** (ConvNeXt @ 125 Hz + iSTFT head, 3.65 M params). Built for a phone-attendant product on **Jetson Nano gen1 CPU** (4× Cortex-A57, ONNX Runtime CPU): the audio leaves through an 8 kHz G.711 channel, so rendering 44.1 kHz wasted ≥80 % of vocoder compute. Voice, prosody, and text front end are **unchanged** — only the decoder is new, distilled from MeloTTS's own output (teacher latent `z` → 8 kHz waveform). Identical input text produces the same speech, just 8 kHz and cheap. ## Results | gate | value | |---|---| | PESQ-NB vs teacher (clean / through G.711 μ-law) | **2.90 / 2.89** | | MCD-DTW (13 MCEP) | 10.8 dB | | Full-model RTF, x86 4-thread ORT 1.17 CPU | **0.013** | | Predicted Jetson Nano A57 RTF (×6–8 host factor) | **0.08–0.10** (was 2–3: unusable) | | Decoder share of full-model compute | 12.6 % (was 88 %) | fp32, ONNX opset 17, no custom ops — runs on the sherpa-onnx-pinned ORT (~1.17) CPU build. **Do not int8-quantize the convs**: measured 9× slower than fp32 on the A57. ## Listen — zh/en code-mixed samples Same text, three renders: the stock teacher at its native 44.1 kHz, the teacher downsampled to 8 kHz (the theoretical best this model could sound), and **this model** rendered end-to-end by the stock sherpa-onnx engine. Small duration differences between teacher and student come from the sherpa C++ frontend vs melo's python frontend (tokenization), not from the vocoder. **1.「您好,這裡是全球科技,幫您轉接給 Kevin 陳經理,請稍候。」** | teacher 44.1 kHz (stock) | teacher @ 8 kHz | **this model (8 kHz, sherpa)** | |---|---|---| | <audio controls src="https://huggingface.co/Luigi/vits-melo-tts-zh_en-8k/resolve/main/samples/sample_1_teacher_44k.wav"></audio> | <audio controls src="https://huggingface.co/Luigi/vits-melo-tts-zh_en-8k/resolve/main/samples/sample_1_teacher_8k.wav"></audio> | <audio controls src="https://huggingface.co/Luigi/vits-melo-tts-zh_en-8k/resolv...