Movie Studio 2.0

Author notes from movieStudio20.json · selected release

###Copy from http -> true this is as download link### Models Required: Wan 2.2 14B FP8 Scaled High Noise Model - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/diffusion_models/wan2.2_t2v_high_noise_14B_fp8_scaled.safetensors?download=true Wan 2.2 14B FP8 Scaled Low Noise Model - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/diffusion_models/wan2.2_t2v_low_noise_14B_fp8_scaled.safetensors?download=true Place in .\models\diffusion_models. Clip Model: umt5_xxl_fp8_e4m3fn_scaled - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors?download=true Place in .\models\text_encoders VAE Model: Wan 2.1 VAE - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/vae/wan_2.1_vae.safetensors?download=true Wan 2.2 VAE - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/vae/wan2.2_vae.safetensors?download=true Place in .\models\vae High Noise Lora (Lightning) - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/loras/wan2.2_t2v_lightx2v_4steps_lora_v1.1_high_noise.safetensors?download=true Low Noise Lora (Lightning) - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/loras/wan2.2_t2v_lightx2v_4steps_lora_v1.1_low_noise.safetensors?download=true Place in .\models\loras *****Copy link and paste these are direct download links***** Information about Group: This is basically an all you need to get started list that will work with my workflow. It is important to note that you can load other High and Low noise Wan models as long as you load the corresponding Clip and VAE. Also I have it preset as a 5 second video at 25 fps. You can change this as well just remember Length = frames per second x number of seconds in the video. I also included the links for the Lightning Loras - this is so you can lower the amount of render step of course you also have to keep the cfg to 1. Basically you can experiment with number of steps but you will get pretty decent results with the default. You can enable or disable the image sharpen node by bypassing it or if you still want a last frame directly connect the noodle from Pull Last Frame and directly connect its Image out noodle to Save Image.

Author notes from movieStudio20.json · selected release

Information about Group: This Group requires no model it is simply load source video - set audio level - load additional audio to be merged - set it's volume - merge. Probably the simplest Group here :)

Author notes from movieStudio20.json · selected release

###Copy from http -> true this is a download link### Setup for this one is a little bit different - go to .\models\tts and add the folder ACE-Step-V1-3.5B. Inside that folder create these 4 folders - DCAE, Vocoder, Transformer, and UMT5. (you can create one for loras as well if you plan to add that node) DCAE- Download these 2 files and put them in the folder DCAE File 1 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/music_dcae_f8c8/config.json?download=true File 2 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/music_dcae_f8c8/diffusion_pytorch_model.safetensors?download=true Vocoder - Download these 2 files and put them in the folder Vocoder File 1 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/music_vocoder/config.json?download=true File 2 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/music_vocoder/diffusion_pytorch_model.safetensors?download=true Transformer - Download these 2 files and put them in the folder Transformer File 1 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/ace_step_transformer/config.json?download=true File 2 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/ace_step_transformer/diffusion_pytorch_model.safetensors?download=true and finally.. UMT5 - Download these 5 files and place it in the UMT5 folder File 1 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/umt5-base/config.json?download=true File 2 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/umt5-base/model.safetensors?download=true File 3 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/umt5-base/special_tokens_map.json?download=true File 4 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/umt5-base/tokenizer.json?download=true File 5 - https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/resolve/main/umt5-base/tokenizer_config.json?download=true *****Copy link and paste these are direct download links***** Information about Group: Should be pretty straightforward - once you have all the files in the right place you are ready to roll. I added a load reference audio which will start out bypassed by default.

Author notes from movieStudio20.json · selected release

###Copy from http -> true this is a download link### So again this setup is a little different - the files are all saved to ./models/mmaudio (you will probably have to create the folder. mmaudio model - https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_large_44k_v2_fp16.safetensors?download=true VAE model - https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_vae_44k_fp16.safetensors?download=true Synchformer - https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_synchformer_fp16.safetensors?download=true Clip model - https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/apple_DFN5B-CLIP-ViT-H-14-384_fp16.safetensors?download=true *****Copy link and paste these are direct download links***** Information about Group: Basically this is used for ambient sounds - door slam, electronic noises essentially anything that is environment related. Speech will not work it will come out as nonsensical gibberish. This audio can be layered in to existing video to give it more atmosphere.

Author notes from movieStudio20.json · selected release

Simple enough each of these tools are modular output from one easily fed into another workflow or group. Because this is modular you can slot in any tools you want I built it specifically to work that way. Enable the tool you want leaving the others disabled. When you have finished running a group tool use the free model and node cache in ComfyUI Manager have fun! Basic Start: 1. Update comfyUI ->./update/update_comfyUi 2. Download FFMPEG from https://www.gyan.dev/ffmpeg/builds/packages/ffmpeg-7.1.1-full_build-shared.7z or any other provider with this version. 3. Create folder FFMPEG and unzip files to that location 4. If you are using Windows (and I assume you are) we need to add paths for Comfyui's Ebeded Python and FFMPEG as both of these will be required to make LatentSync work. 5. To set path in Windows: Win + R and type sysdm.cpl. Click the advanced tab at the top then Environment variables. In the top window select Path and then edit. Click new then browse. Browse to your python_embeded folder and click ok. Click new again and browse then select your FFMPEG/bin folder.Then click ok. Do the same for the bottom window clicking path -> Edit add Python_embeded and FFMPEG. 6. Open ComfyUI and load the workflow. 7. If you get a message to download any files you can either let ComfyUI manager do it or you can use my links. 8. Install all missing nodes. (Takes awhile there are a lot of them) 9. If you don't already have it configured ACE-Step model loader will fail you must setup folders for it and move the required files into them - see the notes.

Author notes from movieStudio20.json · selected release

###Copy from http -> true this is as download link### Models Required: Wan 2.2 14B FP8 Scaled High Noise Model - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/diffusion_models/wan2.2_i2v_high_noise_14B_fp8_scaled.safetensors?download=true Wan 2.2 14B FP8 Scaled Low Noise Model - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/diffusion_models/wan2.2_i2v_low_noise_14B_fp8_scaled.safetensors?download=true Place in .\models\diffusion_models. Clip Model: umt5_xxl_fp8_e4m3fn_scaled - You already have it if you downloaded the requirements for T2V. VAE Model: Wan 2.1 VAE and Wan 2.2 VAE - You already have them if you downloaded the requirements for T2V. High Noise Lora (Lightning) - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/loras/wan2.2_i2v_lightx2v_4steps_lora_v1_high_noise.safetensors?download=true Low Noise Lora (Lightning) - https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/loras/wan2.2_i2v_lightx2v_4steps_lora_v1_low_noise.safetensors?download=true Place in .\models\loras *****Copy link and paste these are direct download links***** Information about Group: This layout is all but a clone of the T2V Group. Once again I set the video output to 5 seconds at 25 frames per second (length 125) at in the T2V workflow. What is great about creating an image of the last frame is it allows for video continuity I added another tool to cleanup that last frame then you can Load your cleaned up image and you are off to the races.

Author notes from movieStudio20.json · selected release

Information about Group: I absolutely love this tool. VibeVoice allows you to take sample audio (and with some fine-tuning) generate any voice saying what you want. Used with LantentSync I am sure you can see the advantages here. The node can be installed using Comfy-UI manager and either model you choose will automatically download for you. Have fun with this one ;)

Author notes from movieStudio20.json · selected release

Information about Group: So my previous version used image sharpen which tended to cause the image to change too much leading to unusual video artifacts and contrast changes. I did a little research and decided to add an upscaler (now that I know what they are) and Image color matching to keep better continuity image to image. You only need 2 files to make this work: skunet color real psnr - https://huggingface.co/deepinv/scunet/resolve/main/scunet_color_real_psnr.pth?download=true - place in your ComfyUI\models\upscale_models folder. codeformer.pth - https://github.com/sczhou/CodeFormer/releases/download/v0.1.0/codeformer.pth - place in your ComfyUI\models\facerestore_models folder.

Author notes from movieStudio20.json · selected release

Setup Instructions: LatentSync error - Torch codec required. This hung me up for several days - but I have the solution for you. Torchcodec and insightface are required to get Lantensync working - first we focus on torch codec: Torchcodec - navigate to your python_embeded folder - click in the location bar and type CMD to open a command prompt. Then run -python -m pip install torchcodec. If you refresh and load again you get an error insightface does not exist. Now I have tried hell and high water to get this to install but try as I might the damn thing refuses to build the wheel - so.. we will just download the built wheel and install it.So here is the trick: First you have to figure out the version on embeded python you have - navigate to .\python_embeded you will see a Zip file called PythonXXX (XXX is your version). Now Download the right Wheel: Python 3.10 - https://huggingface.co/hanamizuki-ai/pypi-wheels/resolve/main/insightface/insightface-0.7.3-cp310-cp310-win_amd64.whl?download=true Python 3.11 - https://huggingface.co/hanamizuki-ai/pypi-wheels/resolve/main/insightface/insightface-0.7.3-cp311-cp311-win_amd64.whl?download=true Python 3.12 https://huggingface.co/hanamizuki-ai/pypi-wheels/resolve/main/insightface/insightface-0.7.3-cp312-cp312-win_amd64.whl?download=true Python 3.13 - https://huggingface.co/MonsterMMORPG/Wan_GGUF/resolve/a200d2eeffa9ba005e3459b16e0efccf218aa137/insightface-0.7.3-cp313-cp313-win_amd64.whl?download=true Download the wheel to your main ComfyUI folder (the one with the run.bat files) click in the navigation bar and open CMD. Run the command - python_embedded\python.exe -m pip install insightface-<version>.whl So mine is 3.13 the command looks like this: python_embedded\python.exe -m pip install insightface-0.7.3-cp313-cp313-win_amd64.whl. And were you to run LatentSync you would get one more error: onnxruntime not installed. So back to the main folder and run either: python.exe -m pip install onnxruntime (for CPU) python.exe -m pip install onnxruntime-gpu (for GPU) Load it up and try running the group - It downloads the models automatically. But not the VAE - See I told you this was a pain. So navigate to .\ComfyUI\custom_nodes\ComfyUI-LatentSyncWrapper create a folder called checkpoints (unless it is already there) then create these folders inside it: Auxiliary and VAE. VAE Download - https://huggingface.co/stabilityai/sd-vae-ft-mse/resolve/main/diffusion_pytorch_model.safetensors?download=true Config Download - https://huggingface.co/stabilityai/sd-vae-ft-mse/resolve/main/config.json?download=true Place these in the VAE folder. Stable_syncnet - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/stable_syncnet.pt?download=true Place it in the main checkpoints folder. Aux - you can download these if you wish: File 1 - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/auxiliary/i3d_torchscript.pt?download=true File 2 - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/auxiliary/koniq_pretrained.pkl?download=true File 3 - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/auxiliary/sfd_face.pth?download=true File 4 - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/auxiliary/syncnet_v2.model?download=true File 5 - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/auxiliary/vgg16-397923af.pth?download=true File 6 - https://huggingface.co/ByteDance/LatentSync-1.6/resolve/main/auxiliary/vit_g_hybrid_pt_1200e_ssv2_ft.pth?download=true Ok everything is in place run the workflow. Aux is optional - https://huggingface.co/ByteDance/LatentSync-1.6/tree/main

Author notes from movieStudio20.json · selected release

###Copy from http -> true this is a download link### Models Required: Wan OVI - https://huggingface.co/Kijai/WanVideo_comfy_fp8_scaled/resolve/main/TI2V/Ovi/Wan2_2-5B-Ovi_960x960_10s_fp8_e4m3fn_scaled_KJ.safetensors?download=true Place in .\models\diffusion_models. UMT5 Model: umt5-xxl-enc-bf16 -https://huggingface.co/Kijai/WanVideo_comfy/resolve/main/umt5-xxl-enc-bf16.safetensors?download=true Place in .\text_encoders MMAudio VAE Models: mmaudio_vae_16k_bf16 -https://huggingface.co/Kijai/WanVideo_comfy/resolve/main/Ovi/mmaudio_vae_16k_bf16.safetensors?download=true place in .\models\vae mmaudio_vocoder_bigvgan_best_netG_bf16 - https://huggingface.co/Kijai/WanVideo_comfy/resolve/main/Ovi/mmaudio_vocoder_bigvgan_best_netG_bf16.safetensors?download=true Place in .\models\vae *****Copy link and paste these are direct download links***** Information about Group: So this is the OVI model minus video. The reason is simple OVI's model can add subtle emotion which Microsoft Vibe Voice cannot. Instead of chewing up all VRAM to create video with audio I tricked the model into thinking it was outputting video - but it never does. Some important notes - First this model is FP16 so using any other model not will result in failure. Second time is not the same as I2V and T2V. Ovi counts every 150 as 5 seconds. So 30 = 1 sec. This will help when trying to decide audio length. Third in order to use the prompt S at the beginning of what someone says E to end it. Audio description can be defined by Audio: For more information - https://github.com/character-ai/Ovi?tab=readme-ov-file