Laionbox Ablation Checkpoints

--- license: apache-2.0 tags: - voice-cloning - text-to-speech - lora - ablation-study - dramabox - ltx-audio --- # LaionBox Ablation Checkpoints LoRA checkpoints and full training/evaluation code from an auxiliary-loss ablation study on **DramaBox** (LTX-2.3 22B audio-only flow-matching TTS model). This repository accompanies the LaionBox voice-cloning fine-tuning project. It contains: - **Trained LoRA adapters** (`.safetensors`) for each ablation condition - **Training metrics** (`metrics.jsonl`) logged every 10 optimizer steps - **Full training and evaluation scripts** under `code/` - **YAML configs** for every ablation condition --- ## Motivation LaionBox v0.1-wip was trained with a differentiable reward LoRA (5 epochs on DramaBox + Emolia data). This ablation study investigates which **non-differentiable auxiliary losses** improve voice quality when combined with the base flow-matching objective. All conditions start from the same v0.1-wip checkpoint and use identical hyperparameters; only the active auxiliary losses differ. --- ## Training Setup | Parameter | Value | |---|---| | **Base model** | LTX-2.3 22B audio-only (`ltx-2.3-22b-dev-audio-only-v13-merged`) | | **Resume from** | LaionBox v0.1-wip (5-epoch diff reward LoRA) | | **LoRA rank / alpha** | 128 / 128 | | **Learning rate** | 2e-5, cosine schedule, 40 warmup steps | | **Effective batch size** | 256 (8 GPUs x 1 micro-batch x 32 grad accum) | | **Duration** | 2 podcast-anchored epochs (~234 optimizer steps) | | **CLAP model** | `gijs/voiceclap-lco-7b-lora` (7B, INT4 quantized) | | **Reward type** | Non-differentiable scalar (no grad through CLAP) | | **Aux trigger** | sigma < 0.4 (~40% of micro-steps) | | **Data mix** | 40% DramaBox, 20% Emolia, 40% Podcast | | **Text dropout** | 10% | ### Auxiliary Losses | Loss | Description | |---|---| | **Naturalness** | CLAP text-audio cosine similarity between generated audio and positive/negative text prompts. Positive: *"Realistic, genuine, spontaneous, authentic..."*. Negative: *"distorted, unnatural, robotic..."* | | **Quality MLP** | P(real) from a binary classifier (MLP head on CLAP embeddings) trained to distinguish real vs. synthetic speech | | **Centroid** | `cos(emb, real_centroid) - cos(emb, synth_centroid)` using pre-computed CLAP embedding centroids | | **Speaker Similarity** | WavLM-SV cosine similarity between reference...