This is old stuff. AnimateDiff was one of the first Video AI solutions for offline needs. And this is the most basic workflow for it. The base for other AnimateDiff workflows.
Runtime profile
Source description
Howto
Adjust prompt. Generate.
Description
This is old stuff. AnimateDiff was one of the first Video AI solutions for offline needs. And this is the most basic workflow for it. The base for other AnimateDiff workflows.
This workflow here is about prompt traveling with AnimateDiff. You choose different prompts for different keyframes. And in the video the prompts morphs into each other.
There are two workflows available. One for AnimateDiff 2 and 3. And one for the AnimateLCM, which is based at AnimateDiff 2.
AnimateDiff and Stable Diffusion 1.5, which are used in conjunction, are trained at 512 pixels in size. So you better won't go this much higher in creation size. The result will become inconsistent. What you can try is to upscale afterwards.
AI-generated commentary
AI-generated explanation based on source and configuration details. Suggestions are clearly labeled.
Use this workflow to make an AnimateDiff video from different prompts assigned to keyframes; the prompts morph into one another over time.
Set different prompts for keyframes, then generate the video.
Choose between a version for AnimateDiff 2 and 3, or one for AnimateLCM, which is based on AnimateDiff 2.
Both versions are described as including upscaling; the example output is a six-second, 1280 × 720 video made with simple AI upscaling.
This version is listed for the SD 1.5 base model and is v1.0 by Arunderan.
Listed model files include analogMadness_v70.safetensors, AnimateLCM_sd15_t2v.ckpt, AnimateLCM_sd15_t2v_lora.safetensors, RealESRGAN_x2.pth, realismBYSTABLEYOGI_v6LCM .safetensors, rife47.pth, v2_lora_ZoomIn.ckpt, v3_sd15_adapter.ckpt, v3_sd15_mm.ckpt, and vae-ft-mse-840000-ema-<
Named node packs include ComfyUI-AnimateDiff-Evolved (ADE_AnimateDiffLoaderWithContext, ADE_AnimateDiffLoRALoader, ADE_AnimateDiffUniformContextOptions, ADE_ApplyAnimateDiffModelSimple, ADE_LoadAnimateDiffModel, ADE_LoopedUniformContextOptions, ADE_MultivalDynamic, ADE_UseEvolved
Other named nodes include CheckpointLoaderSimple, CLIPSetLastLayer, CLIPTextEncode, EmptyLatentImage, Fast Groups Bypasser (rgthree), Fast Groups Muter (rgthree), FreeU, ImageScale, ImageScaleBy, ImageUpscaleWithModel, KSampler, LoraLoaderModelOnly, RIFE VFI, VAEDecode, VAEEncode
The creator says AnimateDiff and SD 1.5 were trained at 512 pixels and advises against going much higher during generation because results may become inconsistent.
Suggestion · not verified
For upscaling, the creator recommends using a separate workflow with a second KSampler.
Suggestion · not verified
The creator says the workflows were made with 8 GB of VRAM, reports frequent out-of-memory errors, and recommends at least 12 GB for proper upscaling.
Suggestion · not verified
No license information is listed for this workflow.
Does this need to be edited?
Sign in to send an edit request.
Sources
1 sourceSource excerpts
2 excerptsSource context: 369 downloads · Type Workflows · Base model SD 1.5
Both workflows comes with upscaling. But i recommend to upscale in an extra workflow. By a second Ksampler. See this article here:
Time
The example with a length of six seconds and a resolution of 1280 x 720 (simple upscaling with AI) finished in two and a half minutes at an 4060 TI with 16gb Vram. It does not use the best upscaling method though.
Requirements
These workflows were generated with 8 gb vram back in the days. However, i ran too often into an OOM. And when you want to do a proper upscaling then i highly suggest a card with at least 12 gb vram.
Estimated VRAM requirement
Estimate unavailable
457 MB across 3 of 7 model files. Model file total + 25% loading overhead + 2 GB execution buffer, rounded up.
Requirements
34 requirementsAnimateLCM_sd15_t2v_lora.safetensors
Not resolvedLoRA · Unknown
AnimateLCM_sd15_t2v.ckpt
Not resolvedCheckpoint · Unknown
Upscaler · 64.0 MB · Hugging Face · ai-forever/Real-ESRGAN
realismBYSTABLEYOGI_v6LCM .safetensors
Not resolvedCheckpoint · Unknown
Checkpoint · Hugging Face
LoRA · 73.9 MB · Hugging Face · guoyww/animatediff
VAE · 319 MB · Hugging Face · stabilityai/sd-vae-ft-mse-original
Node pack · Unknown
Node pack · Unknown
Source context: 353 downloads · Type Workflows · Base model SD 1.5
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
Node pack · Unknown
CheckpointLoaderSimple
PossibleNode pack · Unknown
CLIPSetLastLayer
PossibleNode pack · Unknown
CLIPTextEncode
PossibleNode pack · Unknown
EmptyLatentImage
PossibleNode pack · Unknown
Fast Groups Muter (rgthree)
Not resolvedNode pack · Unknown
FreeU
PossibleNode pack · Unknown
ImageScale
PossibleNode pack · Unknown
ImageScaleBy
PossibleNode pack · Unknown
ImageUpscaleWithModel
PossibleNode pack · Unknown
KSampler
PossibleNode pack · Unknown
LoraLoaderModelOnly
PossibleNode pack · Unknown
RIFE VFI
PossibleNode pack · Unknown
Node pack · Unknown
UpscaleModelLoader
PossibleNode pack · Unknown
VAEDecode
PossibleNode pack · Unknown
VAEEncode
PossibleNode pack · Unknown
VAELoader
PossibleNode pack · Unknown
VHS_GetLatentCount
PossibleNode pack · Unknown
VHS_VideoCombine
PossibleNode pack · Unknown
Node pack · Unknown