ComfyUI workflow for the Union Controlnet Pro from InstantX / Shakker Labs. Available modes: Depth / Pose / Canny / Tile / Blur / Grayscale / Low quality
Perfil de execução
Descrição da fonte
ComfyUI workflow for the Union Controlnet Pro from InstantX / Shakker Labs. Available modes: Depth / Pose / Canny / Tile / Blur / Grayscale / Low quality
Instructions:
Update ComfyUI to the latest version
Download the model to models/controlnet
Load this workflow
Select the correct mode from the SetUnionControlNetType node (above the controlnet loader)
Important : currently need to use this exact mapping to work with the new Union model:
canny - "openpose"
Comentário gerado por IA
Explicação gerada por IA com base nos detalhes da fonte e da configuração. As sugestões são claramente identificadas.
Use this ComfyUI workflow with Flux.1 D to apply Union Controlnet Pro to image generation: provide an input image, choose Depth, Pose, Canny, Tile, Blur, Grayscale, or Low quality, and produce preview and saved-image outputs.
Load it in ComfyUI, provide the input image, and select the mode in the SetUnionControlNetType node above the ControlNetLoader.
To extract control information from your own images, optionally install ControlNet Auxiliary Preprocessors; its openpose, depth, and canny extractors can replace the example image.
Input an image and choose one of seven modes: Depth, Pose, Canny, Tile, Blur, Grayscale, or Low quality.
The graph includes LoadImage, PreviewImage, and SaveImage nodes for loading, previewing, and saving images.
Update ComfyUI, download the Union ControlNet model to models/controlnet, and then load the workflow.
The listed model files are clip_l.safetensors, FLUX.1-dev-ControlNet-Union-Pro/diffusion_pytorch_model.safetensors, flux1-dev-fp8.safetensors, and t5xxl_fp8_e4m3fn.safetensors.
Named nodes include BasicGuider, BasicScheduler, CLIPTextEncode, ControlNetApplySD3, ControlNetLoader, DualCLIPLoader, EmptySD3LatentImage, FluxGuidance, KSamplerSelect, and LoadImage.
Other named nodes include ModelSamplingFlux, OpenposePreprocessor, PreviewImage, RandomNoise, SamplerCustomAdvanced, SaveImage, SetUnionControlNetType, UNETLoader, VAEDecode, and VAELoader.
In SetUnionControlNetType, map canny to openpose, tile to depth, depth to hed/pidi/scribble/ted, and blur to canny/lineart/anime_lineart/mlsd.
Map pose to normal, gray to segment, and low quality to tile in SetUnionControlNetType.
To tune the stated control influence, adjust strength and end_percent in ControlNetApply; the sample values are 0.4 and 0.6, and the stated strength range is 0.3–0.8.
Sugestão · não verificado
Before running, confirm that your ComfyUI setup includes the named nodes used by the workflow.
Sugestão · não verificado
Isso precisa ser editado?
Faça login para enviar uma solicitação de edição.
Fontes
1 fonteTrechos de fonte
2 trechosSource context: 11947 downloads · Type Workflows · Base model Flux.1 D
tile - "depth"
depth - "hed/pidi/scribble/ted"
blur - "canny/lineart/anime_lineart/mlsd"
pose - "normal"
gray - "segment"
low quality - "tile"
So for example, to use the depth mode you would choose "hed/pidi/scribble/ted" in the SetUnionControlNetType node.
Adjust the strength and end_percent in the ControlNetApply node to find a suitable amount of influence. The sample values in the workflow (0.4 and 0.6) are quite gentle, you may want to increase them to have a stronger influence on the image. The model developer has given a recommended range of 0.3-0.8 for strength.
The archive also contains the openpose image used to generate the example image and the original photo the pose was extracted from.
Optional: if you want to extract input information from your own images, install the "ControlNet Auxiliary Preprocessors" custom node. For openpose you can then enable the openpose extractor node (ctrl+b or right click -> Bypass). Auxiliary preprocessors also include extractors for other modes such as depth and canny which you can put in place of the openpose image (remember to set the correct mode afterwards).
Estimativa de requisito VRAM
Estimativa indisponível
16,3 GB distribuídos em 3 de 7 arquivos de modelo. Total de arquivos do modelo + 25% de overhead de carregamento + 2 GB de buffer de execução, arredondado para cima.
Requisitos
Requisitos 24Text encoder · 235 MB · SAFETENSORS · Unknown
DualCLIPLoader
Não resolvidoText encoder · Unknown
ControlNet · Hugging Face · FLUX.1-dev-ControlNet-Union-Pro/diffusion_pytorch_model.safetensors
Checkpoint · 11.5 GB · SAFETENSORS · Unknown
Text encoder · 4.56 GB · SAFETENSOR · fp8 · Unknown
VAEDecode
Não resolvidoVAE · Unknown
VAELoader
Não resolvidoVAE · Unknown
BasicGuider
Não resolvidoPacote de nós · Unknown
BasicScheduler
Não resolvidoPacote de nós · Unknown
CLIPTextEncode
Não resolvidoPacote de nós · Unknown
Source context: 11943 downloads · Type Workflows · Base model Flux.1 D
Pacote de nós · Unknown
ControlNetLoader
Não resolvidoPacote de nós · Unknown
EmptySD3LatentImage
Não resolvidoPacote de nós · Unknown
FluxGuidance
Não resolvidoPacote de nós · Unknown
KSamplerSelect
Não resolvidoPacote de nós · Unknown
LoadImage
Não resolvidoPacote de nós · Unknown
ModelSamplingFlux
Não resolvidoPacote de nós · Unknown
OpenposePreprocessor
Não resolvidoPacote de nós · Unknown
PreviewImage
Não resolvidoPacote de nós · Unknown
RandomNoise
Não resolvidoPacote de nós · Unknown
SamplerCustomAdvanced
Não resolvidoPacote de nós · Unknown
SaveImage
Não resolvidoPacote de nós · Unknown
SetUnionControlNetType
Não resolvidoPacote de nós · Unknown
UNETLoader
Não resolvidoPacote de nós · Unknown