ComfyUI-Reference-Based-Video-Colorization

ComfyUI Reference-Based Video Colorization

A comprehensive ComfyUI implementation featuring two state-of-the-art reference-based video colorization methods:

• 🎨 ColorMNet (2024) - Modern memory-based approach with DINOv2 features
• 🎬 Deep Exemplar (2019) - Classic CVPR method with temporal propagation

Transform black & white videos and images into vibrant color using reference images!

🎬 Demo

See ColorMNet in action colorizing classic black & white footage:

Example: Colorized using ColorMNet with color matching post-processing. The model successfully transfers colors from a reference image while maintaining temporal consistency.

---

✨ Features

ColorMNet Nodes (New)

• ColorMNet Video Colorization - Memory-based temporal coherent colorization
• ColorMNet Image Colorization - Single image colorization
• DINOv2-based feature extraction for superior quality
• Multiple memory modes (balanced, low memory, high quality)
• FP16 support for faster processing
• torch.compile optimization for 15-25% speedup (PyTorch 2.0+)
• Performance reports with timing and FPS metrics

Deep Exemplar Nodes (Original)

• Deep Exemplar Video Colorization - Frame propagation for temporal consistency
• Deep Exemplar Image Colorization - Classic exemplar-based method
• WLS (Weighted Least Squares) filtering for smoother results
• Configurable lambda and sigma parameters
• torch.compile optimization for 15-25% speedup (PyTorch 2.0+)
• SageAttention for 20-30% faster attention operations
• Performance reports for benchmarking

Common Features

• ✅ Automatic model download - No manual setup required!
• ✅ Progress bars - Real-time processing feedback in ComfyUI
• ✅ Performance metrics - Compare speed and quality between methods
• ✅ Flexible resolution - Process at any resolution
• ✅ ComfyUI native - Fully integrated workflow support

---

📦 Installation

Method 1: ComfyUI Manager (Recommended)

1. Install ComfyUI Manager
2. Open ComfyUI → Manager → Install Custom Nodes
3. Search for "Deep Exemplar Video Colorization"
4. Click Install
5. Restart ComfyUI

Models will download automatically on first use (~700MB total).

Method 2: Manual Installation

cd ComfyUI/custom_nodes/
git clone https://github.com/jonstreeter/ComfyUI-Reference-Based-Video-Colorization.git
cd ComfyUI-Reference-Based-Video-Colorization/
pip install -r requirements.txt

Restart ComfyUI. Models download automatically on first use.

Recent changes

• Fixed ColorMNet pipeline normalization to match training (L-only input to encoder, normalized ab masks).
• Added auto-install for git-based CUDA deps (py-thin-plate-spline, correlation sampler) on node load.
• New sample: Workflows/ColorMNet_Image_Workflow.json for single-image colorization.

ColorMNet git-based dependencies

ColorMNet uses two CUDA extensions that are shipped as git repos, not PyPI packages. The node now installs them automatically when it loads, so no manual steps are required. If your environment blocks installs and you need to do it yourself, the commands are:

pip install git+https://github.com/cheind/py-thin-plate-spline.git
pip install git+https://github.com/ClementPinard/Pytorch-Correlation-extension.git

If compilation fails on Windows, install the Desktop development with C++ workload in Visual Studio Build Tools and ensure CUDA_HOME points to your CUDA Toolkit path. The nodes will still run without these extensions, just a bit slower.

---

🚀 Quick Start

Example Workflow