VACE-14B-GGUF AIO ControlNet and Mask Segement — v1.1

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

Since umt5 is a variant based on T5, the input of prompt supports multiple languages. --- 由于 umt5 是基于 T5 的变体，所以提示词输入是支持多种语言输入的

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

Test around with the shift value to get your prefered resutls, you can also disable it.

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

[Tutorial](https://docs.comfy.org/tutorials/video/wan/vace) | [教程](https://docs.comfy.org/zh-CN/tutorials/video/wan/vace) **diffusion_models** - [Wan2.1-VACE-14B-GGUF](https://huggingface.co/QuantStack/Wan2.1-VACE-14B-GGUF) **VAE** - [wan_2.1_vae.safetensors](https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/vae/wan_2.1_vae.safetensors?download=true) **Text encoders** Chose one of following model - [umt5_xxl_fp16.safetensors](https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/text_encoders/umt5_xxl_fp16.safetensors?download=true) - [umt5_xxl_fp8_e4m3fn_scaled.safetensors](https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors?download=true) - [umt5-xxl-encoder-gguf](https://huggingface.co/city96/umt5-xxl-encoder-gguf/tree/main) > You can choose between fp16, fp8 and ggufs. **CausVidLora(Optional for a lot more speed)** - [Wan21_CausVid_14B_T2V_lora_rank32.safetensors](https://huggingface.co/Kijai/WanVideo_comfy/resolve/main/Wan21_CausVid_14B_T2V_lora_rank32.safetensors?download=true) File save location ``` ComfyUI/ ├── models/ │ ├── loras/ │ │ └─── Wan21_CausVid_14B_T2V_lora_rank32.safetensors │ ├── unet/ │ │ └─── Wan2.1-VACE-14B-Qx_x_x.gguf │ ├── text_encoders/ │ │ └─── umt5_xxl_fp8_e4m3fn_scaled.safetensors # or GGUF (umt5-xxl-encoder-Qx_x.gguf) │ └── vae/ │ └── wan_2.1_vae.safetensors ```

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

If you want to speed up your generation by 5x or more, use the causvid lora in the lora loader. If you enable this, you can set the steps to around 4-6 and have to set cfg to 1. That way it can generate a good looking output video in a lot less time.

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

Feel free to enabled those two Optimizations if you have updated torch, and for the one on the left sage attention installed.

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

You can replace this part with another image preprocessor. Related custom node: [comfyui_controlnet_aux](https://github.com/Fannovel16/comfyui_controlnet_aux) 可以把这部分替换成其它的图像预处理器

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

Make sure to use roughly the same frame amount and framerate for your input video, as you do for the generation. Also make sure to set width and height in this node, otherwise you might run into ooms with the canny node. If you replace that part feel free to change this though.

Author notes from vace_v2v_example_workflow_with_reference_image_v1.1.json · selected release

IF you have 32gb but only 12/16gb or even less vram, you might want to use the "Low Vram high ram" loader to offload parts of the model to prevent out of memory errors. You can use this on any card, it can be useful even on a 4090, though be careful with only 16gb ram, you might need to change the virtual vram gb to less.