This is a work flow. The work flows from a movie (which as you may not know is actually just a shitload of pictures) to another movie. Cept this movie sucks to watch. But you can use it as a source for the appropriate ControlNet models, like LTXV IC Control LoRA (which is why I done made it) or for any other reason, you tard. I don't care.
If you do this ahead of time, you don't have to waste resources and/or OOM when cranking out your slop. Prepping your sources usually requires two hands, so maybe wash up first.
Just load a video and select the map of your choice.
Depth is same as regular depth map, it's just a bunch of them. Read the notes, if I put any in there. It's resource intensive and a bit slow, but the map is just the bee's knees.
Pose will give you a funky dancing skeleton. DW Pose NAS is much, much faster than non-NAS.
Canny will give you outlines. Super fast.
They're all useful for certain things. Depth is good when objects are occluding something behind them, but are far away, for example. DW Pose is helpful because, well that's should be obvious. Canny is good for crazy shapes that need outlines, something to define their shape. There is a small group for pre-encoding a latent as well, in case you want to use is as a guide along with the map. Actually no, ignore that. I put it there to fuck with you. Just use the normal audio, which passes directly through and is saved along with the maps. I.e. do nothing. You don't have to do anything. LTXV is EXTREMELY sensitive to audio if you tickle it in the A-spot - try it out, you will be mazed.
Don't forget this part: this is just for making the maps. I did put in some reference nodes to show your stupid brain where to plug in/substitute the saved maps in the actual generation workflow, but you have to connect those dots yourself. Figure it out, eisensteen.
I set up two sizes as defaults. The presets can be changed or added to in the auto size group. Change it to whatever you like to use - depth is the only one that really benefits from going really big.
When using with LTXV I2V your source image MUST be as close as possible to the source motion map first frame. Composition-wise. You will get very strange stuff happening when they don't- even if you turn the strength way down. So what I do is hop over to one of my T2I workflows that has controlnet and generate a batch from the first frame of my maps. This gives you the best chance of an immediate lock and usually avoids having to drop the nightmare-fuel that is mismatched starts. If you're getting horrifying skin monsters all the time, try doing T2V instead until you have good starting frames.
Frame rate is very important here. Make sure that you set the FPS node to match your source. Actually, wait. It's slightly more complicated. If your source is 29.97, you don't want to put that in the FPS node. You would use 30. The VHS loader is NOT forced, if you did force it to treat 29.97 as 30 the count will be messed up. Let it load the native rate and use 30, any discrepancies will be accounted for by the logic - it's usually just one frame +/- Won't be noticable. But if you do it the wrong way it may be, because it might cause duplicate frames or odd dropped ones. So if you're chopping up clips in an editor you don't have to re-encode at a new framerate first, you will just stack errors.
The output FRAME COUNT is set up such that it will always conform to LTXV input rules (mod8 +1). It will round down to the nearest allowed frame count for the model.
I had a batcher in there for doing folders in both, but I took it out for the moment because, shocker, it's not working.
If you put a little effort into preparing your sources properly all of the rest can be automated. Having this done already makes the actual video generation go a LOT faster. Also you can reuse the maps without having to re-map them. Duh.
If you don't have custom nodes, please google "why am I complaining about custom nodes? Am I retarded?" Don't listen to an LLM if it tells you that 'you're super smart and you're doing an awesome job - like, pro move bro - you're on the cutting edge!' I guarantee you that it's lying, hallucinating or probably both.
If you can't figure out how to use this workflow, sell your computer and go live in a box on the street.
