Voxtral 4B TTS 2603 TurboQuant MLX 4bit
> [!TIP] > KV-cache quantization without any fork (recommended, 2026): upstream > llama.cpp/Ollama now cover this natively — use -ctk q80 -ctv q80 > (half KV memory, negligible quality loss: perplexity +0.002–0.05)...
Model source
- Kind
- Other
- File size
- 734 MB–1.40 GB
- Base model
- mistralai/Voxtral-4B-TTS-2603
- Version
- vModel 00001 of 00002
- Updated
- Oct 24, 2026
- Source
- Hugging Face
Source description
[!TIP] KV-cache quantization without any fork (recommended, 2026): upstream llama.cpp/Ollama now cover this natively — use
-ctk q8_0 -ctv q8_0(~half KV memory, negligible quality loss: perplexity +0.002–0.05) or-ctk q4_0 -ctv q4_0(~quarter memory, ≈7.6% perplexity increase). In Ollama:OLLAMA_KV_CACHE_TYPE=q8_0withOLLAMA_FLASH_ATTENTION=1. Keep K and V types symmetric to stay on the fast fused Flash-Attention path. Since April 2026, mainline llama.cpp also applies Hadamard rotation to KV activations (PR #21038), which greatly improves low-bit KV quality (opt-out:LLAMA_ATTN_ROT_DISABLE=1).The RotorQuant/TurboQuant fork flow below is experimental/legacy: the TurboQuant llama.cpp PR was closed without merging (June 2026) and the fork is unmaintained relative to mainline. It is NOT required to use this model.
Voxtral-4B-TTS-2603-TurboQuant-MLX-4bit
4-bit MLX weight-quantized build of mistralai/Voxtral-4B-TTS-2603 with a TurboQuant KV-cache profile. Recommended default for TTS on Apple Silicon.
Hardware compatibility
| Device | VRAM / RAM | Recommendation |
|---|---|---|
| Apple M4 Max 128 GB | ~2.6 GB | recommended — headroom for long context |
| Apple M3 Max 64 GB | ~2.6 GB | comfortable |
| Apple M2 Max 32 GB | ~2.4 GB | fits |
Overview
- Base:
mistralai/Voxtral-4B-TTS-2603— 4B multilingual TTS with zero-shot voice cloning - Weight precision: 4-bit (group-wise)
- KV-cache profile: TurboQuant
- Approx. on-disk size: ~2 GB
- Runtime: MLX on Apple Silicon
Quickstart
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-4bit")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": [
{"type": "audio", "path": "reference_voice.wav"},
{"type": "text", "text": "Hello, this is a cloned voice."},
]}],
add_generation_prompt=True,
)
audio_tokens = generate(model, tokenizer, prompt=prompt, max_tokens=2048)
Model specs
| Field | Value |
|---|---|
| Parameters | 4B |
| Weight bits | 4 |
| Group size | 64 |
| Cache profile | TurboQuant |
| Languages | 9 |
| Voice cloning | Zero-shot |
| Size on disk | ~2 GB |
| Target hardware | Apple Silicon (M1/M2/M3/M4) |
| License | Apache 2.0 |
RotorQuant vs TurboQuant
| TurboQuant | RotorQuant | |
|---|---|---|
| Strategy | Per-head static calibration | Rotational online re-basis |
| Memory reduction | ~3.5x on KV-cache | ~4x on KV-cache |
| Best for | Single-voice sessions | Multi-voice / multi-language batches |
See also
Sources
1 source
Sources
1 sourceVerified Aug 7
Model artifacts
2 artifacts
Model artifacts
2 artifactsSource excerpts
2 excerpts
Source excerpts
2 excerpts--- base_model: mistralai/Voxtral-4B-TTS-2603 library_name: mlx license: apache-2.0 pipeline_tag: text-to-speech tags: - voxtral - audio - speech - tts - text-to-speech - voice-cloning - zero-shot - mlx - turboquant - quantization - 4-bit --- > [!TIP] > **KV-cache quantization without any fork (recommended, 2026):** upstream > llama.cpp/Ollama now cover this natively — use `-ctk q8_0 -ctv q8_0` > (~half KV memory, negligible quality loss: perplexity +0.002–0.05) or > `-ctk q4_0 -ctv q4_0` (~quarter memory, ≈7.6% perplexity increase). In > Ollama: `OLLAMA_KV_CACHE_TYPE=q8_0` with `OLLAMA_FLASH_ATTENTION=1`. Keep > K and V types symmetric to stay on the fast fused Flash-Attention path. > Since April 2026, mainline llama.cpp also applies Hadamard rotation to > KV activations ([PR #21038](https://github.com/ggml-org/llama.cpp/pull/21038)), > which greatly improves low-bit KV quality (opt-out: > `LLAMA_ATTN_ROT_DISABLE=1`). > > The RotorQuant/TurboQuant fork flow below is **experimental/legacy**: the > TurboQuant llama.cpp PR was closed without merging (June 2026) and the fork > is unmaintained relative to mainline. It is NOT required to use this model. <!-- kv-upstream-note --> # Voxtral-4B-TTS-2603-TurboQuant-MLX-4bit 4-bit MLX weight-quantized build of [`mistralai/Voxtral-4B-TTS-2603`](https://huggingface.co/mistralai/Voxtral-4B-TTS-2603) with a TurboQuant KV-cache profile. Recommended default for TTS on Apple Silicon. ## Hardware compatibility | Device | VRAM / RAM | Recommendation | | --- | --- | --- | | Apple M4 Max 128 GB | ~2.6 GB | recommended — headroom for long context | | Apple M3 Max 64 GB | ~2.6 GB | comfortable | | Apple M2 Max 32 GB | ~2.4 GB | fits | ## Overview - **Base:** `mistralai/Voxtral-4B-TTS-2603` — 4B multilingual TTS with zero-shot voice cloning - **Weight precision:** 4-bit (group-wise) - **KV-cache profile:** TurboQuant - **Approx. on-disk size:** ~2 GB - **Runtime:** MLX on Apple Silicon ## Quickstart ```bash pip install mlx-lm ``` ```python from mlx_lm import load, generate model, tokenizer = load("majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-4bit") prompt = tokenizer.apply_chat_template( [{"role": "user", "content": [ {"type": "audio", "path": "reference_voice.wav"}, {"type": "text", "text": "Hello, this is a cloned voice."}, ]}], add_generation_prompt=True, ) audio_tokens = generate(model, tokenizer,...
Source context: 109 downloads · 0 likes · Pipeline text-to-speech · Library mlx · Repo majentik/Voxtral-4B-TTS-2603-TurboQuant-MLX-4bit
