Hunyuanvideo 1.5 Robotwin15 I2V LoRA

--- license: other base_model: hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v_step_distilled library_name: diffusers pipeline_tag: image-to-video tags: - diffusers - lora - hunyuanvideo-1.5 - image-to-video - robotics - robotwin --- # HunyuanVideo 1.5 RobotWin-15 I2V LoRA checkpoints This public RoMALab repository contains HunyuanVideo 1.5 four-step I2V LoRA checkpoints post-trained on 15 RobotWin tasks. The comparison runs use stride-16 temporal coverage, rank-16 video attention Q/K/V/O adapters, video FFN adapters, mixed spatial gating, a future-latent MeanFlow objective, and global batch 16 over four GPUs. ## August 2026 conditioned and wrist-view runs These are the latest checkpoints from the four requested RobotWin-15 post-training experiments. All use global batch 16, stride-16 temporal windows, minimum future arm-joint path 0.05, and a configured 15,000-step budget; the published step is the final checkpoint reached by each allocation. | Checkpoint | Step | Video attention | Video FFN | Spatial gate | Explicit conditioning / wrist path | |---|---:|---:|---:|---:|---| | `r64-ffn-r16-gate75-jointmotion005-step-011080` | 11,080 | r64 | r16 | 75% | none | | `task-arm-r16-ffn-r32-step-013571` | 13,571 | r16 | r32 | off | 15-task + left/right/both one-hot | | `task-arm-wrist-siglip-r16-ffn-r32-step-012700` | 12,700 | r16 | r32 | off | task/arm + pooled SigLIP token concatenation | | `task-arm-wrist-resnet-xattn-r16-ffn-r32-step-012995` | 12,995 | r16 | r32 | off | task/arm + ResNet-50 cross-attention at blocks 17/35/53 | Each folder contains `config.yaml`, `resolved_config.json`, `metadata.json`, `conditioning.pt`, LoRA safetensors, and the resumable training state. `metadata.json` records byte sizes and SHA-256 hashes. For the three explicitly conditioned variants, `conditioning.pt` is required for inference; the LoRA file alone does not contain the task/arm or wrist-fusion parameters. ## Completed July 2026 runs | Checkpoint | Step | Training | Video attention | Video FFN | Eligible-sample gate probability | Minimum future arm-joint path | Trainable parameters | W&B | |---|---:|---|---:|---:|---:|---:|---:|---| | `ffn-r32-gate75-step-015000` | 15,000 | resumed from 9,279; complete | r16 | r32 | 75% | none | 49,545,216 | [run](https://wandb.ai/joeclinton1/video2vla-hunyuan-posttraining/runs/3pifc2yn) | | `ffn-r16-gate75-step-015000`...

--- license: other base_model: hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v_step_distilled library_name: diffusers pipeline_tag: image-to-video tags: - diffusers - lora - hunyuanvideo-1.5 - image-to-video - robotics - robotwin --- # HunyuanVideo 1.5 RobotWin-15 I2V LoRA checkpoints This public RoMALab repository contains HunyuanVideo 1.5 four-step I2V LoRA checkpoints post-trained on 15 RobotWin tasks. The three July 2026 comparison runs use stride-16 temporal coverage, rank-16 video attention Q/K/V/O adapters, video FFN adapters, mixed spatial gating, a future-latent MeanFlow objective, and global batch 16 over four GPUs. ## Latest comparison checkpoints | Checkpoint | Step | Video attention | Video FFN | Eligible-sample gate probability | Trainable parameters | W&B | |---|---:|---:|---:|---:|---:|---| | `ffn-r32-gate75-step-009279` | 9,279 | r16 | r32 | 75% | 49,545,216 | [run](https://wandb.ai/joeclinton1/video2vla-hunyuan-posttraining/runs/kmc3idkh) | | `ffn-r16-gate75-step-009000` | 9,000 | r16 | r16 | 75% | 31,850,496 | [run](https://wandb.ai/joeclinton1/video2vla-hunyuan-posttraining/runs/8zbwq20e) | | `ffn-r16-gate20-step-010419` | 10,419 | r16 | r16 | 20% | 31,850,496 | [run](https://wandb.ai/joeclinton1/video2vla-hunyuan-posttraining/runs/jx3k12cp) | The gate probability applies only to samples with reviewed spatial-gate metadata; other samples remain ungated. The step-9,000 rank-16/gate-75 checkpoint was saved successfully before a later distributed SIGABRT terminated that job. Each folder contains: ```text config.yaml metadata.json lora/pytorch_lora_weights.safetensors training_state.pt ``` The LoRA file is sufficient for inference. `training_state.pt` contains optimizer and learning-rate-scheduler state for exact-topology continuation; PyTorch checkpoint files should only be loaded from trusted sources. ## Loading ```python from diffusers import HunyuanVideo15Transformer3DModel transformer = HunyuanVideo15Transformer3DModel.from_pretrained( "hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v_step_distilled", subfolder="transformer", ) transformer.load_lora_adapter( "RoMALab/hunyuanvideo-1.5-robotwin15-i2v-lora", subfolder="checkpoints/ffn-r16-gate20-step-010419/lora", adapter_name="robotwin", ) ``` ## Legacy checkpoint `checkpoint-006000/` is retained for reproducibility. It is the earlier attention-onl...

Source context: 0 downloads · 0 likes · Pipeline image-to-video · Library diffusers · Repo RoMALab/hunyuanvideo-1.5-robotwin15-i2v-lora