KVAE 2D 1.0

--- library_name: KVAE 2D tags: - model_hub_mixin - pytorch_model_hub_mixin - vae license: mit --- <div align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kandinskylab/kvae-1/refs/heads/main/assets/kvae-white.png"> <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/kandinskylab/kvae-1/refs/heads/main/assets/kvae-black.png"> <img alt="Shows an illustrated sun in light mode and a moon with stars in dark mode." src="https://user-images.githubusercontent.com/25423296/163456779-a8556205-d0a5-45e2-ac17-42d089e3c3f8.png"> </picture> </div> <div align="center"> <a href="https://github.com/kandinskylab/kvae-1">Github</a> | <a href="https://habr.com/ru/companies/sberbank/articles/966450/">Habr</a> | <a href="https://kandinskylab.ai/">Project Page</a> | Technical Report (soon) </div> <h1>KVAE 1.0: Image tokenizer</h1> KVAE-2D model has compression 8x8 and 16 latent channels. ## Evaluation results <img src="assets/kvae2d-comparison.png" /> Evaluation results of KVAE-2D model on [Imagenet-256](https://huggingface.co/datasets/benjamin-paine/imagenet-1k-256x256) (valid) and [DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/) (valid, high-resolution). All compared models perform 8x8 compression with 16 latent channels: | Dataset | Model | PSNR | SSIM | LPIPS | rFID | |---------------------|---------|-----------|----------|-----------|----------| | ImageNet (256, val) | Wan-2.1 | 29.03 | 0.85 | 0.069 | 0.62 | | | ImageNet (256, val) | Flux | 31.11 | **0.91** | **0.041** | **0.11** | | ImageNet (256, val) | KVAE 2D | **31.71** | **0.91** | 0.054 | 0.46 | | DIV2K | Wan-2.1 | 31.87 | 0.89 | 0.069 | - | | | DIV2K | Flux | 32.64 | 0.91 | 0.061 | - | | DIV2K | KVAE 2D | **33.67** | **0.92** | **0.060** | - |