FLUX - Advanced Redux with ControlNet [Flux-Tool] — v1.0

Author notes from REDUX universal.json · selected release

Short background on Redux Redux works in two steps. First there is a Clip Vision model that crops your input image into square aspect ratio and reduce its size to 384x384 pixels. It splits this image into 27x27 small patches and each patch is projected into CLIP space. Redux itself is just a very small linear function that projects these clip image patches into the T5 latent space. The resulting tokens are then added to your T5 prompt. Intuitively, Redux is translating your conditioning input image into "a prompt" that is added at the end of your own prompt. So why is Redux dominating the final prompt? It's because the user prompt is usually very short (255 or 512 tokens). Redux, in contrast, adds 729 new tokens to your prompt. This might be 3 times as much as your original prompt. Also, the Redux prompt might contain much more information than a user written prompt that just contains the word "anime". https://github.com/kaibioinfo/ComfyUI_AdvancedRefluxControl - check out this link if you want more info.

Author notes from REDUX universal.json · selected release

My feeling currently is that downsampling by far works best. So I would first try downsampling with 1-3 and only use the other options if the effect is too weak or too strong.

Author notes from REDUX universal.json · selected release

**Style transfer** 1) Disable IMG-2 and IMG-3 BLOCKs. 2) Set two times "false" for IMG2 and IMG3 conditionings. 3) Write down in manual simple prompt desribing your style, like "anime", "cartoon", "pixel art" etc... 4) Simply by choosing between different modes in florence2 you can simplify or add more details in image, but sometimes it might override your "style" prompt so feel free to experiment or just bypass autoprompt.

Author notes from REDUX universal.json · selected release

ReduxAdvanced This node allows for more customization. As input it gets the conditioning (prompt), the Redux style model, the CLIP vision model and optionally(!) the mask. Its parameters are: "downsampling_factor": This is the most important parameter and it determines how strongly the conditioning image influences the generated image. The strength value from 1 (strongest), to 9 (weakest), meaning closer to 1 = more similarities with input image. "downsampling_function": These functions are the same as in any graphics program when you resize an image. The default is "area", but "bicubic" and "nearest_exact" are also interesting choices. The chosen function can very much change the outcome, so its worth experimenting a bit. "mode": How the image should be cropped: center crop (square) is what Redux is doing by default: cropping your image to a square image and then resizing it to 384x384 pixel keep aspect ratio will add a padding to your image such that it gets square. It will adjust the mask (or generate one if no one is given) such that the padding is not part of the mask. Note that the final image is still resized to 384x384, so your image will be compressed more if you add more padding. autocrop with mask will crop your image in a way that the masked area is in the center. It will also crop your image such that only the masked area and a margin remains. The margin is specified as the autocrop_margin parameter and is relative to the total size of the image. So autocropping with a margin of 0.1 means the crop is done on the masked area + 10% of the image on each side. --In that wflow you can use masks for img2 and img3 but make sure to bypass rescaler for those images or you can just crop the image.-- "weight": This option downscales the Redux tokens by the given value squared. This is very similar to the "conditioning average" approach many people use to reduce the effect of Redux on the generated image. This is an alternative way of reducing Redux' impact, but downsampling works better in most cases. However, feel free to also experiment with this value, or even with a combination of both: downsampling AND weight. "autocrop_margin" this parameter is only used when "autocrop with mask" is selected as mode.

Author notes from REDUX universal.json · selected release

You can gain additional control between conditionings weights by bypassing different parts of the workflow like autoprompt or redux. Or by choosing different caption mode.

Author notes from REDUX universal.json · selected release

THIS IS VERY IMPORTANT! "IMG1+IMG2 Conditioning Mixer" and "IMG2+IMG3 Conditioning Mixer" SET "true" 2 times for 3 images - conditioning gets prompt from all 3 images. SET "true" in "IMG1+IMG2 Conditioning Mixer" and "false" in "IMG2+IMG3 Conditioning Mixer" for 2 images - conditioning gets prompt from IMG1 and IMG2. SET "false" 2 times for 1 image - prompt only for IMG1 conditioning. If you forget abbout this settings, it'll still work, but every time it'll send empty tokens so image will have slight difference. Changing inputs between 1 and 2 allows you to use average weights instead of concat. In most cases concat seems to work better, i added this for additional controls in case you need it.

Author notes from REDUX universal.json · selected release

You can set input to 2, if you want to use conditioning average. Closer to 0 = Conditioning from Florence2 prompt will have higher priority than redux and vice versa. Setting "false" here means that redux won't be blending with promt conditionings, this will also emulate weight average 1.00 which is by definition basic redux behavior Basically, if you set "true" this will give more weight to prompt conditionings = more prompt adharance