--- license: apache-2.0 base_model: - Lightricks/LTX-2.3 tags: - video-generation - lora - ltx-video - dual-character - dialogue - cinematic - chinese-drama - image-to-video pipeline_tag: image-to-video language: - en - zh --- # LTX-Video 2.3 — Dual-Character LoRA (English mirror) A field-tested **image-to-video character-consistency LoRA** for `Lightricks/LTX-2.3` (22B distilled), tuned for two-character dialogue scenes and multi-shot cinematic video generation. > ⚠️ **Naming note (corrected 2026-05-21):** > The original filename and ModelScope repo include the string "IC-LORA", but **this is NOT an IC-LoRA** in the strict technical sense (parallel-canvas / `video_conditioning` mechanism). An A/B/C test (same prompt + seed, three reference-channel variants) confirmed that the LoRA's actual conditioning mechanism is **first-frame pixel pinning** (the regular i2v path), not parallel-canvas attention. Earlier copy on this card incorrectly described it as IC-LoRA — that has been removed. Credit to ZKong for raising the discrepancy in the discussions tab. --- ## Example renders Episode is an 8-shot Chinese palace drama (《玉佩定情》 + 《暗夜阴谋》) with three characters: 沈月华 (Shen Yuehua, heroine), 萧云霄 (Xiao Yunxiao, prince), 慕容静 (Murong Jing, antagonist). Render config: 1280×704, 121 frames @ 24 fps, ambient audio. ### Single-character identity — Shen Yuehua walking in the garden, picks up a jade pendant ### Dual-character dialogue — Shen + Xiao meet (the LoRA's signature use case) ### Cross-scene identity — Murong Jing in a different location (palace night chamber) ### Three-character composition — the LoRA's upper limit --- ## What this LoRA does Fine-tuned on `Lightricks/LTX-2.3` (22B distilled), specifically for: 1. **Two-character dialogue scenes** — significantly reduces character drift when two people appear in the same frame 2. **Cinematic shot composition** — reinforced for dialogue-driven framing (close-up ↔ medium ↔ wide) 3. **Multi-shot narrative continuity** — better understanding of multi-segment prompts (storyboard-style descriptions) 4. **Style compatibility** — works well across 古风仙侠 (ancient Chinese fantasy), 现代都市 (modern urban), and 3D 动漫 styles The reference image is consumed via **first-frame pixel pin** (standard i2v conditioning), not via the parallel-canvas / `video_conditioning` channel. --- ## How to use (correct pattern) ### Single-character shot ```python # Upstream LTX-2.3 distilled pipeline — single reference as first-frame pin from ltx_pipelines.distilled import DistilledPipeline from ltx_pipelines.utils.args import ImageConditioningInput from ltx_core.loader import LoraPathStrengthAndSDOps, sd_ops as _sd_ops_mod import torch lora = LoraPathStrengthAndSDOps( "LTX2.3-IC-LORA-Dual-Character.safetensors", 0.8, # strength (standalone) _sd_ops_mod.LTXV_LORA_COMFY_RENAMING_MAP, ) pipe = DistilledPipeline( distilled_checkpoint_path="ltx-2.3-22b-distilled-1.1.safetensors", spatial_upsampler_path="ltx-2.3-spatial-upscaler-x2-1.1.safetensors", gemma_root="google/gemma-3-12b-it-qat-q4_0-unquantized", loras=[lora], device=torch.device("cuda:0"), ) video, audio = pipe( prompt="...", seed=42, height=704, width=1280, num_frames=121, # 5 s @ 24 fps, satisfies 8k+1 frame_rate=24, images=[ImageConditioningInput( # first-frame pin = THE reference mechanism path="character_ref.png", frame_idx=0, strength=0.9, )], enhance_prompt=False, ) ``` ### Dual-character shot LTX's i2v pin rejects two pins at the same `frame_idx`, so two refs can't both be pinned at frame 0. Two workable patterns: **Pattern A (recommended): composite reference image.** Build one image with character A on the left and character B on the right (e.g., via PIL `Image.paste` or any image editor), pin THAT at `frame_idx=0`. Both identities transfer in one pin. **Pattern B: stagger the pins.** Pin character A at frame 0, character B at a later latent boundary (e.g., frame 64 — must be a multiple of 8 per the VAE's temporal compression). Only works if B doesn't need to be visible from the very first frame. ### Recommended parameters | Setting | Value | |---|---| | Resolution | 1280 × 704 (16:9, native LTX-2.3 distilled training resolution) | | Faster preview | 960 × 544 (~40% faster, slightly less detail) | | Frames | satisfy 8k+1 — e.g. 121 (5 s), 193 (8 s), 241 (10 s), 361 (15 s) at 24 fps | | Strength | Standalone 0.7-0.9 · stacked with style LoRAs 0.3-0.5 | | Pin strength | 0.85-0.95 for tight identity, 0.7 for looser "inspired-by" | | Trigger word | None | --- ## Field-tested production tips Quirks of this LoRA + the LTX-2.3 distilled backbone that aren't in the original card but matter in practice. ### 1. Repeat color tokens for dark-clothed characters This LoRA has a light-wuxia-robe bias. Dark outfits drift toward white at low pin strength. **Repeat the color token glued to each clothing noun**: ```text BAD: black fedora and black suit GOOD: BLACK fedora, white shirt, BLACK suit jacket, BLACK trousers, ... BLACK suit, BLACK trousers throughout ``` Also bump pin strength to ~0.95 for color fidelity on dark outfits. ### 2. **Never use quoted dialogue in prompts** This LoRA was trained on Chinese drama clips with burned-in Chinese subtitles. **Any quoted dialogue (`「...」` or `"..."`) in the prompt causes the LoRA to hallucinate subtitle characters at the bottom of the frame.** Single biggest gotcha. ```text BAD: 低声警告 「此茶不可饮!」 ← fake on-screen subtitles GOOD: 低声急切警告她茶水有毒 ← clean output, indirect narration ``` If your app needs subtitles, burn them post-hoc via `ffmpeg drawtext`. ### 3. Avoid "object detaches" prompts during action At high motion intensity, the model loses object tracking. A directive like "fedora flies off mid-spin and tumbles to the floor" produces broken output — the hat dematerialises. Either: - Keep the object attached and say so explicitly ("the fedora STAYS ON his head throughout the spin") - Or render attach + detach as two clips and concat ### 4. Cross-shot identity drift For multi-shot dialogue scenes, character identity drifts across cuts. Workaround: re-pin the reference image at frame 0 of every shot. (Deterministic seed + same first-frame pin + same prompt scaffolding produces good repeatability.) ### Render performance - **Resolution:** 1280 × 704, 121 frames @ 24 fps (~5 s output) - **Hardware:** NVIDIA A800 80 GB → ~70 s per shot - **Output:** mp4 with ambient audio track (no TTS) On consumer hardware (RTX 4090 24 GB), expect ~3-4 minutes per shot. --- ## Limitations 1. **Subtitle hallucination** with quoted dialogue (see tip #2) 2. **Complex physical interactions** (wrestling, hugging, intricate hand-on-hand) can deform 3. **Tail-frame artifact** of LTX-2.3 — last 6-8 frames may smear; trim post-hoc if needed 4. **Action complexity ceiling** — the 8-step distilled budget caps motion complexity at action peaks 5. **Portrait orientation** degrades identity (LoRA trained on landscape only) 6. **Dual-character via two separate refs is awkward** (see "How to use" above) — composite-image pin is the cleanest workaround --- ## Original Chinese README (preserved) The original Chinese model card from ModelScope is reproduced below for users who want the unmodified original documentation. (Note: the original card uses the "IC-LoRA" label — the term has been kept here for fidelity, even though the A/B/C test described above shows the conditioning mechanism is first-frame i2v pinning rather than parallel-canvas IC-LoRA.) 点击展开原版中文模型卡片 (click to expand original Chinese README) ### LTX-Video (2.3) IC-LoRA: 双人分镜头对话增强模型 本模型是基于 Lightricks LTX-2.3 底模训练的 IC-LoRA,专为双人同框对话、角色互动及分镜头视频生成场景深度优化。 **一、模型核心提升** 1. 角色参考稳定性:显著提升双人同框时的人物特征一致性,减少角色漂移。 2. 分镜构图稳定性:针对影视化对话构图进行了加固,支持更精准的镜头控制。 3. 叙事连贯性:增强了对多段描述的理解力,使分镜间的过渡衔接更自然。 4. 风格兼容性:完美支持古风仙侠、现代都市、3D 动漫等主流视觉风格。 **二、模型基本信息** 1. 基础模型:Lightricks/LTX-2.3 2. 许可证:Apache-2.0 3. 管道标签:image-to-video, text-to-video 4. 模型用途:仅供学习交流使用 5. 开发者:麻雀 AI **三、运行指南** 1. 推荐平台:ComfyUI 2. 支持工作流:ComfyUI 官方 LTX 工作流、KJ-LTX 插件工作流 3. 生成模式:文生视频 (T2V) 与 图生视频 (I2V) 均支持 4. 硬件参考:RTX 5090 显卡在 720P 分辨率下,单条视频生成耗时约 2 分钟 **四、推荐参数配置** 1. 分辨率:建议使用 16:9 (如 1280x720) 2. 时长与帧率:建议时长 ≥10 秒,帧率设定为 24 FPS 3. LoRA 权重设定: - 独立使用建议:0.6 - 1.0 - 叠加其他 LoRA 使用时建议:0.3 - 0.5 **五、Prompt 编写规范** 1. 编写逻辑:需包含完整的场景描述 + 角色设定 + 分镜设计 + 镜头语言,强化双人对话互动逻辑。 2. 触发词说明:无需特定触发词。 **六、效果说明与局限性** 1. 优势风格:在古风、现代、3D 动漫类双人对话场景中表现最佳。 2. 已知限制:受限于 LTX-2.3 底模性能,极其复杂的双人肢体互动(如缠绕、打斗)可能出现形变。 3. 运动幅度:建议以对话和微动作为主,大动态动作的连贯性仍有提升空间。 --- ## Hardware requirements | GPU | VRAM | Works? | |---|---|---| | A100 / A800 80 GB | 80 GB | ✅ ~70 s per 5 s shot | | RTX 4090 / 3090 | 24 GB | ✅ ~3-4 min per 5 s shot | | RTX 4080 / 4070 Ti Super | 16 GB | ❌ won't fit 22B in bf16 | | anything This is an English-language mirror of [fxj1131's LTX2.3 Dual-Character LoRA on ModelScope](https://www.modelscope.cn/models/fxj1131/LTX2.3-IC-LORA-Dual-Character). > All credit for the model weights belongs to the original author, **麻雀 AI (Maque AI)**. > This mirror exists to make the model + documentation accessible to HuggingFace users who cannot easily access ModelScope, and to share field-tested usage notes from a production deployment. > **The `.safetensors` weights file is unmodified and byte-identical to the ModelScope upload.** --- ## License Apache License 2.0 — same as the original. See `LICENSE` and `NOTICE`.
