ControlNet Anime Colorize

--- license: mit datasets: - SubMaroon/danbooru-lineart base_model: - cagliostrolab/animagine-xl-3.0 --- # Experimental ControlNet (Low Quality / Research Prototype) > **Experimental model. Low quality. Not intended for production use.** > This ControlNet was trained as a research experiment to explore line-based conditioning and colorization behavior in SDXL anime models. --- ## Model Summary This repository contains an **experimental ControlNet for SDXL**, trained on anime-style images. The model is **not stable**, shows **inconsistent color behavior**, and should be treated as a **research prototype** rather than a finished or polished solution. The goal of this experiment was to understand: - How SDXL ControlNet learns **colorization from line-based conditioning** - How different conditioning types (Canny vs Lineart) affect **color consistency** --- ## Base Model - **Base model:** `cagliostrolab/animagine-xl-3.0` - **Architecture:** ControlNet SDXL - **Training framework:** 🤗 Diffusers - **Precision:** `bf16` --- ## Conditioning Type - Primary conditioning: **Lineart / Canny-like edges** - Backgrounds are mostly white - Line quality varies (mostly clean, some noisy samples) > Important limitation: > Lineart / Canny **does not contain color information**, which leads to unstable and drifting color predictions. --- ## Dataset - Size: ~**14,000 image pairs** - Format: - Original image (color) - Conditioning image (lineart / canny) - Prompt (caption) ### Known dataset issues - Some lineart images are **noisy or inconsistent** - Images are resized to square resolution (possible cropping artifacts) - No explicit color supervision - No palette or region-level color constraints --- ## Training Configuration Typical training setup: ```bash resolution: 768 train_batch_size: 2 gradient_accumulation_steps: 2 effective_batch_size: 4 learning_rate: 2e-5 lr_scheduler: cosine max_train_steps: 6000–8000 mixed_precision: bf16