Showing how to do video to video in comfyui and keeping a consistent face at the end.
Runtime profile
Source description
Showing how to do video to video in comfyui and keeping a consistent face at the end.
We keep the motion of the original video by using controlnet depth and open pose.
We use animatediff to keep the animation stable.
Finally ReActor and face upscaler to keep the face that we want. Optionally we also apply IPAdaptor during the generation to help keep the face closer to what we want even before the swap. (Note: IPAdaptor can greatly slow down the generation depending on your machine)
Estimated VRAM requirement
Estimate unavailable
4.51 GB across 3 of 8 model files. Model file total + 25% loading overhead + 2 GB execution buffer, rounded up.
AI-generated commentary
AI-generated explanation based on source and configuration details. Suggestions are clearly labeled.
Use this ComfyUI workflow with an original video and a face you want to keep to create a video-to-video face swap that carries the original video's motion into the output video.
ControlNet depth and open pose carry motion from the original video into the result.
AnimateDiff is used to keep animation stable, while ReActor and a face upscaler keep the desired face in the result.
The original video supplies the motion, and the desired face is the one to retain in the resulting video.
Open the Civitai listing for “Video to Video with Face Swap in ComfyUI — v1.0” by A1lazydog: https://civitai.com/models/161796?modelVersionId=182135.
Required ComfyUI node packs include ComfyUI-ReActor, ComfyUI-Impact-Pack, ComfyUI-AnimateDiff-Evolved, and comfyui_controlnet_aux.
Other required node packs are was-node-suite-comfyui, ComfyUI_FizzNodes, ComfyUI-Advanced-ControlNet, and facerestore_cf.
IPAdapter is optional during generation and can help keep the generated face closer to your desired face before the swap.
IPAdapter can greatly slow generation, depending on your machine.
Does this need to be edited?
Sign in to send an edit request.
Sources
1 sourceSource excerpts
1 excerptSource context: 7892 downloads · Type Workflows · Base model Other
Requirements
35 requirementscontrol_depth-fp16.safetensors
Not resolvedControlNet · Unknown
control_openpose-fp16.safetensors
Not resolvedControlNet · Unknown
GFPGANv1.4.pth
Not resolvedCheckpoint · Unknown
inswapper_128.onnx
Not resolvedCheckpoint · Unknown
ip-adapter-plus-face_sd15.bin
Not resolvedLoRA · Unknown
Checkpoint · 1.99 GB · Hugging Face · digiplay/Juggernaut_final
Checkpoint · 1.69 GB · Hugging Face · guoyww/animatediff
Vision model · 850 MB · Hugging Face · google-t5/t5-base
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
CheckpointLoaderSimple
PossibleNode pack · Unknown
CLIPTextEncode
PossibleNode pack · Unknown
CLIPVisionLoader
PossibleNode pack · Unknown
ControlNetApplyAdvanced
PossibleNode pack · Unknown
Node pack · Unknown
DWPreprocessor
PossibleNode pack · Unknown
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
IPAdapterApply
Not resolvedNode pack · Unknown
IPAdapterModelLoader
PossibleNode pack · Unknown
LoadImage
PossibleNode pack · Unknown
Node pack · Unknown
PrepImageForClipVision
PossibleNode pack · Unknown
PreviewImage
PossibleNode pack · Unknown
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
VAEDecode
PossibleNode pack · Unknown
VAEEncode
PossibleNode pack · Unknown
VHS_LoadVideo
PossibleNode pack · Unknown
VHS_VideoCombine
PossibleNode pack · Unknown