Steerable-Motion

Wan

The Wan approach uses VACE to create anchor images and continuations from previous images, which are chained together at the end:

Main example

Sample workflow for Wan

You can find a workflow here to get started.

Animatediff

The Animatediff approach uses a combination of IP-Adapter and SparseCtrl to travel between images:

Main example

5 basic workflows for Animatediff

Below are 5 basic workflows - each with their own weird and unique characteristics - all with differing levels of adherence and different types of motion - most of the changes come from tweaking the IPA configuration and switching out base models:

• Smooth n' Steady: tends to have nice smooth motion - good starting point
• Rad Attack: probably the best for realistic motion
• Slurshy Realistiche: moves in a slightly realistic manner but is a little bit slurshy
• Chocky Realistiche: realistic-ish but very blocky
• Liquidy Loop: smooth and liquidy

You can see each in acton below:

basic workflows

Philosophy for getting the most from these

This isn't an approach like text to video that will perform well out of the box, it's more like a paint brush - an artistic tool that you need to figure out how to get the best from.

Through trial and error, you'll need to build an understanding of how the motion and settings work, what its limitations are, which inputs images work best with it, etc.

It won't work for everything but if you can figure out how to wield it, this approach can provide enough control for you to make beautiful things that match your imagination precisely.

In both cases, tweaking the settings can greatly influence the motion - for example, below you can see two examples of the same images animated - but with the one setting tweaked, the length of each frame's influence:

Tweaking settings example

Want to give feedback, or join a community who are pushing open source models to their artistic and technical limits?

You're very welcome to drop into our Discord here.

Credits

For Animatediff, the code draws heavily from Cubiq's IPAdapter_plus, while the workflow uses Kosinkadink's Animatediff Evolved and ComfyUI-Advanced-ControlNet, Fizzledorf's Fizznodes, Fannovel16's Frame Interpolation and more.

For Wan, it's built on top of the work of Kijai's wonderful [ComfyUI-WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrap

Verified Sep 12