ModelPrésentéExterne

Laionbox v0.6 Wip

LoRA fine-tune of the DramaBox audio-only DiT (LTX-2.3-22B-Dev, 3.3B, ResembleAI) for expressive voice-acting TTS. Pure LoRA (rank 256 / alpha 256, 5 epochs) — no AdaLN, no auxiliary losses, just flow matching....

Sourcehuggingface.co/ChristophSchuhmann/laionbox-v0.6-wip/tree/3754230d68b95fcae96f39b1230b99ef3fe368a2

TypeLoRAVersionlora_r256_step2465CClasse

Source du modèle

Type: LoRA
Taille du fichier: 1,69 GB
Version: vlora_r256_step2465
Mis à jour: 20 oct. 2026
Téléchargements: 0
Source: Hugging Face

Description de la source

LAIONBox v0.6-wip — Run 15 (LoRA-only, rank 256)

LoRA fine-tune of the DramaBox audio-only DiT (LTX-2.3-22B-Dev, ~3.3B, ResembleAI) for expressive voice-acting TTS. Pure LoRA (rank 256 / alpha 256, 5 epochs) — no AdaLN, no auxiliary losses, just flow matching. Scaled up from the v0.5 baseline (r128 / 3 epochs).

Evaluation (means over 72 samples; sidon_normalized enhanced refs: 5 speakers x 2 seeds x 6 prompts EN+DE)

nisqa_mos/scoreq/CE/PQ are held-out scorers. 🔊 Audio: see v06_eval_results.html (base64 players) in this repo.

Verdict: run5 (full fine-tune + AdaLN, v0.3-wip) has the best raw quality; run15 (this model, LoRA r256) has the best speaker similarity (raw 0.92, vc->sidon 0.947) while matching the r128 baseline on quality.

RAW

model	mos	utmos	nisqa_mos	scoreq	CE	PQ	spk_sim
Run 5 — full-FT DiT + AdaLN (v0.3)	4.601	3.647	4.369	4.636	6.188	7.896	0.879
Run 10 — LoRA r128 baseline (v0.5)	4.481	3.448	4.201	4.5	5.982	7.793	0.876
Run 12 — frozen DiT + AdaLN (best)	4.378	3.395	4.057	4.428	6.023	7.858	0.894
Run 13 — frozen DiT + AdaLN (best)	4.434	3.498	4.138	4.471	5.997	7.789	0.88
Run 15 — LoRA r256 5ep (s2440)	4.474	3.385	4.184	4.487	6.0	7.851	0.919
Run 15 — LoRA r256 5ep (s2460)	4.487	3.418	4.205	4.504	6.003	7.841	0.916
Run 15 — LoRA r256 5ep (s2465)	4.491	3.446	4.199	4.512	6.0	7.841	0.916

SIDON

model	mos	utmos	nisqa_mos	scoreq	CE	PQ	spk_sim
Run 5 — full-FT DiT + AdaLN (v0.3)	4.747	3.793	4.667	4.74	6.273	7.921	0.87
Run 10 — LoRA r128 baseline (v0.5)	4.656	3.588	4.545	4.647	6.116	7.912	0.873
Run 12 — frozen DiT + AdaLN (best)	4.616	3.537	4.488	4.606	6.168	7.962	0.89
Run 13 — frozen DiT + AdaLN (best)	4.654	3.659	4.528	4.645	6.12	7.911	0.877
Run 15 — LoRA r256 5ep (s2440)	4.648	3.553	4.543	4.622	6.133	7.947	0.918
Run 15 — LoRA r256 5ep (s2460)	4.661	3.603	4.56	4.636	6.132	7.941	0.915
Run 15 — LoRA r256 5ep (s2465)	4.649	3.596	4.55	4.634	6.13	7.94	0.914

VC->SIDON

model	mos	utmos	nisqa_mos	scoreq	CE	PQ	spk_sim
Run 5 — full-FT DiT + AdaLN (v0.3)	4.713	3.823	4.575	4.683	6.223	7.908	0.92
Run 10 — LoRA r128 baseline (v0.5)	4.647	3.729	4.533	4.639	6.082	7.905	0.92
Run 12 — frozen DiT + AdaLN (best)	4.579	3.65	4.477	4.589	6.126	7.963	0.919
Run 13 — frozen DiT + AdaLN (best)	4.622	3.767	4.5	4.625	6.093	7.933	0.923
Run 15 — LoRA r256 5ep (s2440)	4.643	3.646	4.526	4.605	6.088	7.932	0.946
Run 15 — LoRA r256 5ep (s2460)	4.638	3.643	4.501	4.606	6.078	7.924	0.948
Run 15 — LoRA r256 5ep (s2465)	4.651	3.696	4.524	4.617	6.099	7.943	0.947

Training

Base: ltx-2.3-22b-dev-audio-only-v13-merged (DramaBox audio-only DiT)
LoRA rank 256, alpha 256 (scaling 1.0), dropout 0.0; pure flow-matching loss; 5 epochs / 2465 steps; lr 1e-4 cosine; bf16; 8xA100
Checkpoint: lora_r256_step2465.safetensors (453M params)

Inference

LoRA adapter for the DramaBox DiT — pass only the LoRA (no --adaln-checkpoint):

python inference_adaln.py \
  --checkpoint ltx-2.3-22b-dev-audio-only-v13-merged.safetensors \
  --full-checkpoint ltx-2.3-22b-dev.safetensors \
  --lora-checkpoint lora_r256_step2465.safetensors \
  --prompt "A warm, slightly husky 35-year-old woman, high-quality studio recording. 'I never thought I would say this out loud.'" \
  --voice-sample reference_speaker.wav --output out.wav --seed 42

Standalone: drop --voice-sample, add --no-ref, use a full prompt. Post-process with Sidon; for max speaker-sim use Chatterbox-VC -> Sidon.

Sources

1 source

Hugging FacePrincipalVersion 3754230d68b95fcae96f39b1230b99ef3fe368a2

huggingface.co/ChristophSchuhmann/laionbox-v0.6-wip/tree/3754230d68b95fcae96f39b1230b99ef3fe368a2

Vérifié 30 août

Artefacts du modèle

1 artefact

lora_r256_step2465.safetensors

safetensors · 1,69 GB · SHA-256 ee07d9fabc98…3ea5 · Hugging Face

Download file

Extraits de sources

2 extraits

Readme30 aoûthuggingface.co/ChristophSchuhmann/laionbox-v0.6-wip

--- license: other tags: [text-to-speech, tts, voice-acting, dramabox, lora, laionbox] --- # LAIONBox v0.6-wip — Run 15 (LoRA-only, rank 256) LoRA fine-tune of the **DramaBox audio-only DiT** (LTX-2.3-22B-Dev, ~3.3B, ResembleAI) for expressive voice-acting TTS. Pure LoRA (rank 256 / alpha 256, 5 epochs) — **no AdaLN, no auxiliary losses**, just flow matching. Scaled up from the v0.5 baseline (r128 / 3 epochs). ## Evaluation (means over 72 samples; **sidon_normalized** enhanced refs: 5 speakers x 2 seeds x 6 prompts EN+DE) `nisqa_mos`/`scoreq`/`CE`/`PQ` are **held-out** scorers. 🔊 Audio: see `v06_eval_results.html` (base64 players) in this repo. **Verdict:** **run5** (full fine-tune + AdaLN, v0.3-wip) has the **best raw quality**; **run15** (this model, LoRA r256) has the **best speaker similarity** (raw 0.92, vc->sidon 0.947) while matching the r128 baseline on quality. ### RAW | model | mos | utmos | nisqa_mos | scoreq | CE | PQ | spk_sim | |---|---|---|---|---|---|---|---| | Run 5 — full-FT DiT + AdaLN (v0.3) | 4.601 | 3.647 | 4.369 | 4.636 | 6.188 | 7.896 | 0.879 | | Run 10 — LoRA r128 baseline (v0.5) | 4.481 | 3.448 | 4.201 | 4.5 | 5.982 | 7.793 | 0.876 | | Run 12 — frozen DiT + AdaLN (best) | 4.378 | 3.395 | 4.057 | 4.428 | 6.023 | 7.858 | 0.894 | | Run 13 — frozen DiT + AdaLN (best) | 4.434 | 3.498 | 4.138 | 4.471 | 5.997 | 7.789 | 0.88 | | Run 15 — LoRA r256 5ep (s2440) | 4.474 | 3.385 | 4.184 | 4.487 | 6.0 | 7.851 | 0.919 | | Run 15 — LoRA r256 5ep (s2460) | 4.487 | 3.418 | 4.205 | 4.504 | 6.003 | 7.841 | 0.916 | | Run 15 — LoRA r256 5ep (s2465) | 4.491 | 3.446 | 4.199 | 4.512 | 6.0 | 7.841 | 0.916 | ### SIDON | model | mos | utmos | nisqa_mos | scoreq | CE | PQ | spk_sim | |---|---|---|---|---|---|---|---| | Run 5 — full-FT DiT + AdaLN (v0.3) | 4.747 | 3.793 | 4.667 | 4.74 | 6.273 | 7.921 | 0.87 | | Run 10 — LoRA r128 baseline (v0.5) | 4.656 | 3.588 | 4.545 | 4.647 | 6.116 | 7.912 | 0.873 | | Run 12 — frozen DiT + AdaLN (best) | 4.616 | 3.537 | 4.488 | 4.606 | 6.168 | 7.962 | 0.89 | | Run 13 — frozen DiT + AdaLN (best) | 4.654 | 3.659 | 4.528 | 4.645 | 6.12 | 7.911 | 0.877 | | Run 15 — LoRA r256 5ep (s2440) | 4.648 | 3.553 | 4.543 | 4.622 | 6.133 | 7.947 | 0.918 | | Run 15 — LoRA r256 5ep (s2460) | 4.661 | 3.603 | 4.56 | 4.636 | 6.132 | 7.941 | 0.915 | | Run 15 — LoRA r256 5ep (s2465) | 4.649 | 3.596 | 4.55 | 4.634 | 6.13 | 7.94 | 0.914 |...

Note30 aoûthuggingface.co/ChristophSchuhmann/laionbox-v0.6-wip

Source context: 0 downloads · 1 likes · Pipeline text-to-speech · Repo ChristophSchuhmann/laionbox-v0.6-wip