FireRedTTS3-ComfyUI

FireRedTTS3-ComfyUI

English | 中文

Version: v0.2.5

ComfyUI nodes for FireRedTeam/FireRedTTS3: zero-shot voice cloning across 24 languages and 21 Chinese dialects, instruction-based voice design, semantic + acoustic speech editing, Whisper reference transcription, and ComfyUI/AIMDO DynamicVRAM support.

ComfyUI Hugging Face Hugging Face License

License note: FireRedTTS3 is released by the FireRed Team under Apache-2.0 for academic research purposes. Do not use voice cloning without consent.

Features

• Native in-process inference - The Qwen3 backbone, PatchEncoder, DiT flow head, RedAE codec, and CAM++ speaker encoder run directly inside ComfyUI; no external server, no remote code.
• Two model variants - fireredtts3_base (zero-shot cloning with language tags) and fireredtts3_instruct (cloning + voice design + speech editing), sharing one RedAE codec.
• 24 languages + 21 Chinese dialects - Explicit language/dialect dropdown, or auto with FastText lid.176 detection (heuristic zh/ja/en fallback).
• Voice design - Describe a voice in natural language (gender, age, timbre, pace, accent) and synthesize it; the model's voice plan (CoT) is returned as text.
• Speech editing - Semantic editing (insert/delete/replace words, returns the rewritten text) and acoustic editing (speed 0.5-2.0x, pitch ±6 steps, volume 0.3-2.0x) via the trained instruction templates.
• Text frontend included - Local wetext normalization for Chinese/English (numbers, dates, units to spoken form), sentence splitting with cross-fade joins. No external API calls.
• ComfyUI AUDIO in/out - Reference voices, edit inputs, and generated audio all use standard ComfyUI AUDIO.
• Optimized flow stepping - timestep embeddings, per-block AdaLN modulations, and rotary frequencies are computed once per generation instead of once per patch, and the generation buffers are preallocated: ~11-16% faster with bit-identical output (verified by same-seed A/B).
• AIMDO DynamicVRAM support - Core, codec, and speaker encoder are registered as separate ComfyUI models with castable weights, paging through ComfyUI/AIMDO when DynamicVRAM is active.
• bf16 mirror - Optional half-size weights with official-equivalent mixed precision: the backbone LLM and RedAE encoder are stored in bf16 (the official code already computes them under bf16 autocast), while the flow head and RedAE decoder stay fp32. Same-seed A/B testing against the official fp32 weights produced identical waveforms (cosine 1.0000, SNR > 80 dB).
• Whisper transcription node - Turns a reference clip into the prompt_text transcript that noticeably improves cloning.
• INT8 ConvRot weights (experimental) - the FireRedTTS3-int8 mirror ships the core transformer linears pre-quantized to Comfy INT8 ConvRot (format int8_tensorwise, per-row fp32 scales, offline Hadamard weight rotation, group size 256) using the official comfy-kitchen quantizer. The loader auto-detects *.comfy_quant keys and executes through comfy_kitchen.int8_linear(convrot=True) with online activation rotation; everything else (embeddings, boundary projections, stop_head, Conv1d, RedAE, CAM++) stays float. 321/332 linears -> checkpoint 7.90 -> 3.07 GiB, with validated output quality.