Tem um workflow do ComfyUI? Gere uma documentação legível direto no navegador.Torne um workflow legível.Experimentar agora ↗
A image to video ComfyUI workflow with CogVideoX. Tested with CogvideoX Fun 1.1 and 1.5. Note that the motion lora does not work with the Fun 1.5 model. Just with the 1.1 one.
Perfil de execução
Descrição da fonte
A image to video ComfyUI workflow with CogVideoX. Tested with CogvideoX Fun 1.1 and 1.5. Note that the motion lora does not work with the Fun 1.5 model. Just with the 1.1 one.
The zipfile contains the json file, the starter image, and the png file from creation, which also contains the workflow.
This workflow also contains a CogVideoX motion lora for the camera movement. And you can also add further instructions in the prompt. CogVideoX relies at motion informations in text form.
It also has a very simple upscaling method implemented. I am at my journey to figure out a special upscaling workflow though. But for some it might still be useful. It is super fast compared to an upsampling by another ksampler.
CogVideo creation size is limited. The old version 1.1 is fixed to a 16:10 format. And 720x480 resolution. The new version 1.5 goes up to double size, but the motion lora that i use here does not work with it.
Comentário gerado por IA
Explicação gerada por IA com base nos detalhes da fonte e da configuração. As sugestões são claramente identificadas.
Use this ComfyUI workflow to turn a starter image and prompt text with motion instructions into a CogVideoX video, with a simple upscaling path.
It includes a CogVideoX motion LoRA for camera movement and a simple upscaling method.
Input: a starter image and prompt text, which can include additional movement instructions.
Output: a generated video, with simple upscaling available in the workflow.
The archive contains the JSON workflow, the starter image, and a PNG from creation that also contains the workflow.
Expand collapsed Note nodes for model links, placement information, and other notes.
The workflow lists these model files or model IDs: alibaba-pai/CogVideoX-Fun-V1.1-5b-InP, microsoft/Florence-2-large, 4x-UltraSharp.pth, pytorch_lora_weights.safetensors, rife47.pth, and t5xxl_fp8_e4m3fn.safetensors.
Named ComfyUI nodes include CLIPLoader, CogVideoDecode, CogVideoImageEncodeFunInP, CogVideoLoraSelect, CogVideoSampler, CogVideoTextEncode, Crop (mtb), DF_Get_image_size, DownloadAndLoadCogVideoModel, DownloadAndLoadFlorence2Model, Fast Groups Bypasser (rgthree), Florence2Run, ۽.
Other named nodes include ImageScale, ImageUpscaleWithModel, Int Literal, JWImageMix, JWIntegerMul, LoadImage, RIFE VFI, ShowText|pysssss, TextCombinations, UpscaleModelLoader, VHS_VideoCombine, and VRAM_Debug.
Add movement instructions to the prompt; CogVideoX uses motion information in text form.
Sugestão · não verificado
The listed size limits are 720x480 in 16:10 for Fun 1.1, while Fun 1.5 goes up to double size.
The included motion LoRA does not work with Fun 1.5; verify that you are using Fun 1.1 if you need that LoRA.
Sugestão · não verificado
Before use, verify that pytorch_lora_weights.safetensors, VHS_VideoCombine, CogVideoLoraSelect, and RIFE VFI are available. The description reports that 8 GB was too low, says 12 GB should work, and says 12 GB was not tested.
Sugestão · não verificado
Isso precisa ser editado?
Faça login para enviar uma solicitação de edição.
Fontes
1 fonteTrechos de fonte
1 trechoSource context: 1839 downloads · Type Workflows · Base model CogVideoX
There are some collapsed Note nodes besides the important nodes. Click at them to expand them.
The Note nodes contain further informations. And in case of the models also links to the models, and where to put them.
This is my first upload at Civitai, so suggesstions and feedback is very welcome.
EDIT: The example video rendered in 8 minutes plus some overhead for preparation, without the upscaling with CogvideoX Fun version 1.1. at an 4060 TI. Version 1.5 renders doube as fast. But the motion lora does not work. Upscaling is another 4 minutes then if i remember right. I was not able to run CogvideoX at my old card with 8 Gb. That's too low. 12 gb should work. But i cannot test it. I render at a card with 16 gb at the moment.
And there is an explanation video at Youtube:
Estimativa de requisito VRAM
Estimativa indisponível
4,62 GB distribuídos em 2 de 8 arquivos de modelo. Total de arquivos do modelo + 25% de overhead de carregamento + 2 GB de buffer de execução, arredondado para cima.
Requisitos
Requisitos 30Upscaler · 63.9 MB · PT · Unknown
alibaba-pai/CogVideoX-Fun-V1.1-5b-InP
Não resolvidoCheckpoint · Hugging Face · alibaba-pai/CogVideoX-Fun-V1.1-5b-InP
CLIPLoader
Não resolvidoText encoder · Unknown
ImageUpscaleWithModel
Não resolvidoUpscaler · Unknown
Vision model · Hugging Face · microsoft/Florence-2-large · Florence 2 Large
pytorch_lora_weights.safetensors
PossívelLoRA · Unknown
Text encoder · 4.56 GB · SAFETENSOR · fp8 · Unknown
UpscaleModelLoader
Não resolvidoUpscaler · Unknown
CogVideoDecode
Não resolvidoPacote de nós · Unknown
CogVideoImageEncodeFunInP
Não resolvidoPacote de nós · Unknown
CogVideoLoraSelect
Não resolvidoPacote de nós · Unknown
Pacote de nós · Unknown
CogVideoTextEncode
Não resolvidoPacote de nós · Unknown
Crop (mtb)
Não resolvidoPacote de nós · Unknown
DF_Get_image_size
Não resolvidoPacote de nós · Unknown
DownloadAndLoadCogVideoModel
Não resolvidoPacote de nós · Unknown
DownloadAndLoadFlorence2Model
Não resolvidoPacote de nós · Unknown
Fast Groups Bypasser (rgthree)
Não resolvidoPacote de nós · Unknown
Florence2Run
Não resolvidoPacote de nós · Unknown
ImageResizeKJ
Não resolvidoPacote de nós · Unknown
ImageScale
Não resolvidoPacote de nós · Unknown
Int Literal
Não resolvidoPacote de nós · Unknown
JWImageMix
Não resolvidoPacote de nós · Unknown
JWIntegerMul
Não resolvidoPacote de nós · Unknown
LoadImage
Não resolvidoPacote de nós · Unknown
RIFE VFI
Não resolvidoPacote de nós · Unknown
ShowText|pysssss
Não resolvidoPacote de nós · Unknown
TextCombinations
Não resolvidoPacote de nós · Unknown
VHS_VideoCombine
Não resolvidoPacote de nós · Unknown
VRAM_Debug
Não resolvidoPacote de nós · Unknown
