Wan 2.2 TI2V 5b ControlNet Hed v1

--- license: apache-2.0 language: - en tags: - video - video-generation - video-to-video - controlnet - diffusers - wan2.2 --- # Controlnet for Wan2.2 (hed) This repo contains the code for controlnet module for Wan2.2. See <a href="https://github.com/TheDenk/wan2.2-controlnet">Github code</a>. Same approach as controlnet for [Wan2.1](https://github.com/TheDenk/wan2.1-dilated-controlnet). <video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/63fde49f6315a264aba6a7ed/l_gD3__VP6HhMgqmmNS35.mp4"></video> ### For ComfyUI Use the cool [ComfyUI-WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper). <img width="600" height="480" alt="t2i_workflow" src="https://github.com/user-attachments/assets/4788c2d0-8ff0-405e-9b6d-0e0b1347865b" /> The latest version of the `diffusers` is required. You can install it like this: ```bash pip install git+https://github.com/huggingface/diffusers.git ``` ### Inference examples #### Simple inference with cli ```bash python -m inference.cli_demo \ --video_path "resources/bubble.mp4" \ --prompt "Close-up shot with soft lighting, focusing sharply on the lower half of a young woman's face. Her lips are slightly parted as she blows an enormous bubblegum bubble. The bubble is semi-transparent, shimmering gently under the light, and surprisingly contains a miniature aquarium inside, where two orange-and-white goldfish slowly swim, their fins delicately fluttering as if in an aquatic universe. The background is a pure light blue color." \ --controlnet_type "hed" \ --base_model_path Wan-AI/Wan2.2-TI2V-5B-Diffusers \ --controlnet_model_path TheDenk/wan2.2-ti2v-5b-controlnet-hed-v1 ``` #### Minimal code example ```python import os os.environ['CUDA_VISIBLE_DEVICES'] = "0" os.environ["TOKENIZERS_PARALLELISM"] = "false" import torch from diffusers.utils import load_video, export_to_video from diffusers import AutoencoderKLWan, UniPCMultistepScheduler from controlnet_aux import HEDdetector from wan_controlnet import WanControlnet from wan_transformer import CustomWanTransformer3DModel from wan_t2v_controlnet_pipeline import WanTextToVideoControlnetPipeline base_model_path = "Wan-AI/Wan2.2-TI2V-5B-Diffusers" controlnet_model_path = "TheDenk/wan2.2-ti2v-5b-controlnet-hed-v1" vae = AutoencoderKLWan.from_pretrained(base_model_path, subfolder="vae", torch_dtype=torch.float32) transformer = CustomWa...