ComfyUI-SamplerSchedulerMetricsTester

Sampler Scheduler Metrics Tester for ComfyUI

This custom node for ComfyUI allows you to test combinations of samplers and schedulers automatically. It generates a batch of generated images(latents), as well as RGB images, each one annotated with the specific combination used, performance timing, and several image quality metrics (Laplacian Variance, Gradient Mean, FFT-based sharpness) for our internal usage, but kept as they could may be useful for others to. Have you ever wanted to have an easy way to render all combinations of samplers and schedulers with zero manual work? Then this custom node could help you. Header image

Features

• Iterate trough all samples and schedulers
• Iterate through specified lists of samplers and schedulers.
• Overlay generated images with:
  • Image Number: Image: X/Y (#Z) format (1-based count / total, and 0-based index).
  • Sampler and Scheduler names.
  • Seed used for the test.
  • Generation time.
  • Several image quality metrics:
    • Laplacian Variance (sharpness).
    • Gradient Mean (detail/edges).
    • FFT-based Sharpness Score.
  • Status (OK/ERROR) and error messages if issues occur.
• Customizable font size, text color, and text background for overlays.
• Outputs a batch of generated latents, a batch of annotated images, and a text summary of all tests.

Installation

1. Clone this repository or download the folder containing these files (e.g., sampler_scheduler_metrics_tester_project) into your ComfyUI/custom_nodes/ directory.

   cd ComfyUI/custom_nodes/
   git clone https://github.com/IcelandicCenterArtificialIntelligence/ComfyUI-SamplerSchedulerMetricsTester.git
   

2. Dependencies: This node uses opencv-python (cv2) for image metrics. If you don't have it installed in your ComfyUI's Python environment, you'll need to install it:

   pip install opencv-python
   

3. Restart ComfyUI.

How to Use

1. In ComfyUI, add the "Sampler Scheduler Metrics Tester" node (found under the "sampling/testing" category).
2. Connect the required inputs:
   • model: The diffusion model.
   • vae: The VAE model.
   • positive: Positive conditioning.
   • negative: Negative conditioning.
   • latent_image: An initial latent image (e.g., from an "Empty Latent Image" node).
3. Adjust parameters:
   • seed, steps, cfg, denoise: Standard sampling parameters. The main seed is used to initialize the noise pattern for the entire batch of tests.
   • sampler_list_override: (Optional) Enter a comma or newline-separated list of samplers to test (e.g., euler, dpm_2, dpm_adaptive). Leave empty to use all available samplers. Names are case-insensitive but will use the official casing.
   • scheduler_list_override: (Optional) Enter a comma or newline-separated list of schedulers (e.g., normal, karras, simple). Leave empty for all available.
   • font_size_ratio, text_color, text_bg_color: Customize overlay text appearance.
4. The node will iterate through all valid combinations, generating an image for each one.

Example Workflow

Below is an example of how the "Sampler Scheduler Metrics Tester" node can be used in a workflow.

1. Load the Example Workflow:

You can easily load the example workflow into ComfyUI using one of the following methods:

• Drag & Drop Workflow PNG: Click the image below to download workflow.png. This PNG image has the entire ComfyUI workflow embedded within it. Simply download the workflow.png file and then drag and drop it directly onto your ComfyUI canvas to load the workflow.

  Example Workflow for Sampler Scheduler Metrics Tester

• Load Workflow JSON: Alternatively, you can download the workflow as a standard JSON file: Download example_workflow.json
  Once downloaded, click the "Load" button in you