Loading
Du hast einen ComfyUI-Workflow? Erstelle direkt im Browser eine verständliche Workflow-Dokumentation.Workflow verständlich machen.Jetzt ausprobieren ↗
A fine-tuned version of Qwen/Qwen3-TTS-12Hz-1.7B-Base for Saudi Arabian (Khaleeji/KSA) Arabic speech synthesis.
Modellquelle
Quellenbeschreibung
A fine-tuned version of Qwen/Qwen3-TTS-12Hz-1.7B-Base for Saudi Arabian (Khaleeji/KSA) Arabic speech synthesis.
Training data: — ~13 k Arabic utterances in the KSA dialect, filtered to 1–20 s duration.
Quellen
1 QuelleVerifiziert 22. Sept.
Modellartefakte
1 Artefaktcheckpoint-epoch-0/model.safetensors
safetensors · 3,57 GB · SHA-256 5aafb408086f…e1d8 · Hugging Face
HerunterladenQuellenauszüge
2 Auszüge
vadimbelsky/KSA_Arabic_English_Dataset_13kThe base model ships with a fixed set of languages in its codec token vocabulary; Arabic was not among them. Adding it required changes at three levels:
Arabic was assigned codec token ID 2072. Rather than initialising this embedding randomly, it was set to the mean of all existing language embeddings before training:
ARABIC_LANG_ID = 2072
codec_emb = qwen3tts.model.talker.model.codec_embedding
existing_ids = [v for k, v in config.talker_config.codec_language_id.items() if k != 'arabic']
avg = codec_emb.weight[existing_ids].float().mean(0)
codec_emb.weight[ARABIC_LANG_ID] = avg
A 4-token block injects the explicit language ID through the codec channel:
pos 3: codec_think_id
pos 4: codec_think_bos_id
pos 5: lang_id ← Arabic token 2072
pos 6: codec_think_eos_id
pos 7: speaker embedding slot ← shifted +1 vs. base model
The sequence offset in the collator was adjusted from +8 to +9, and codec_embedding_mask[7] = False so the speaker embedding is injected directly from the speaker encoder.
dataset.py detects Arabic automatically from Unicode range \u0600–\u06FF, so no explicit language field is needed per sample.
Speaker ID 3000 (ksa_speaker) was registered. The embedding is extracted from a reference KSA audio clip by the frozen speaker encoder and written directly into the safetensors weights — the saved model is fully self-contained.
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen3-TTS-12Hz-1.7B-Base |
| Training data | vadimbelsky/KSA_Arabic_English_Dataset_13k (Arabic subset) |
| Optimizer | AdamW, lr=2e-6, weight decay=0.01 |
| Precision | bf16 mixed precision |
| Gradient accumulation | 4 steps (effective batch ~32) |
| Gradient clipping | 1.0 |
| Epochs | 5 (this checkpoint: epoch 4) |
| Loss | talker_loss + 0.3 × sub_talker_loss |
All model parameters were fine-tuned (no LoRA). The speaker encoder was kept frozen during training.
Install dependencies:
pip install qwen-tts soundfile torch
Single utterance:
import torch
import soundfile as sf
from qwen_tts.inference.qwen3_tts_model import Qwen3TTSModel
tts = Qwen3TTSModel.from_pretrained(
"vadimbelsky/qwen3-TTS-KSA",
dtype=torch.bfloat16,
device_map="cuda:0",
attn_implementation="sdpa",
)
wavs, sr = tts.generate_custom_voice(
text="الحين سويت فنجال قهوة، توني صحيت من النوم",
speaker="ksa_speaker",
language="arabic",
)
sf.write("output.wav", wavs[0], sr)
CLI with infer_ksa.py:
python infer_ksa.py \
--checkpoint vadimbelsky/qwen3-TTS-KSA \
--text "وين تبي تلتقي الحين؟" \
--output out_ksa.wav
Output is 24 kHz mono WAV.
| Speaker name | Language |
|---|---|
ksa_speaker | arabic |
Apache 2.0 — same as the base model.
--- language: - ar license: apache-2.0 base_model: Qwen/Qwen3-TTS-12Hz-1.7B-Base pipeline_tag: text-to-speech tags: - tts - text-to-speech - arabic - saudi - ksa - fine-tuned - qwen3 datasets: - vadimbelsky/KSA_Arabic_English_Dataset_13k --- # Qwen3-TTS — KSA Arabic Fine-tune A fine-tuned version of [`Qwen/Qwen3-TTS-12Hz-1.7B-Base`](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) for **Saudi Arabian (Khaleeji/KSA) Arabic** speech synthesis. Training data: [`vadimbelsky/KSA_Arabic_English_Dataset_13k`](https://huggingface.co/datasets/vadimbelsky/KSA_Arabic_English_Dataset_13k) — ~13 k Arabic utterances in the KSA dialect, filtered to 1–20 s duration. --- ## How Arabic support was added The base model ships with a fixed set of languages in its codec token vocabulary; Arabic was not among them. Adding it required changes at three levels: ### 1. Arabic language embedding — warm-start initialisation Arabic was assigned codec token ID `2072`. Rather than initialising this embedding randomly, it was set to the **mean of all existing language embeddings** before training: ```python ARABIC_LANG_ID = 2072 codec_emb = qwen3tts.model.talker.model.codec_embedding existing_ids = [v for k, v in config.talker_config.codec_language_id.items() if k != 'arabic'] avg = codec_emb.weight[existing_ids].float().mean(0) codec_emb.weight[ARABIC_LANG_ID] = avg ``` ### 2. Language-conditioned codec prefix (4-token think block) A 4-token block injects the explicit language ID through the codec channel: ``` pos 3: codec_think_id pos 4: codec_think_bos_id pos 5: lang_id ← Arabic token 2072 pos 6: codec_think_eos_id pos 7: speaker embedding slot ← shifted +1 vs. base model ``` The sequence offset in the collator was adjusted from `+8` to `+9`, and `codec_embedding_mask[7] = False` so the speaker embedding is injected directly from the speaker encoder. ### 3. Language auto-detection `dataset.py` detects Arabic automatically from Unicode range `\u0600`–`\u06FF`, so no explicit language field is needed per sample. ### 4. KSA speaker registration Speaker ID `3000` (`ksa_speaker`) was registered. The embedding is extracted from a reference KSA audio clip by the frozen speaker encoder and written directly into the safetensors weights — the saved model is fully self-contained. --- ## Training setup | Setting | Value | |---|---| | Base model | `Qwen/Qwen3-TTS-...
Source context: 10 downloads · 0 likes · Pipeline text-to-speech · Repo TTS-ORG/mans_model_mostafa