Txt2Img MHN VQGAN

--- license: mit tags: - vae - autoencoder - remote-sensing - image-generation - vqgan - diffusers - pytorch pipeline_tag: image-to-image --- # Txt2Img-MHN-VQGAN VQGAN model for remote sensing image generation, part of the [Txt2Img-MHN](https://arxiv.org/abs/2208.04441) framework. ## Model Details - **Class:** `VQModel` (diffusers) - **Input/Output:** 3-channel RGB images (256×256) - **Latent channels:** 256 - **Codebook:** 1024 embeddings (256-dim) - **Parameters:** ~72M ## Usage ```python from diffusers import VQModel import torch # Load model vae = VQModel.from_pretrained( "BiliSakura/Txt2Img-MHN-VQGAN", ignore_mismatched_sizes=True ) # Encode image to latent image = torch.randn(1, 3, 256, 256) with torch.no_grad(): latent = vae.encode(image).latents # (1, 256, 32, 32) # Decode latent to image with torch.no_grad(): decoded = vae.decode(latent).sample # (1, 3, 256, 256) ``` ## Training Trained on the [RSICD](https://github.com/201528014227051/RSICD_optimal) remote sensing dataset using Taming Transformers. ## Citation ```bibtex @article{txt2img_mhn, title={Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networks}, author={Xu, Yonghao and Yu, Weikang and Ghamisi, Pedram and Kopp, Michael and Hochreiter, Sepp}, journal={IEEE Trans. Image Process.}, doi={10.1109/TIP.2023.3323799}, year={2023} } ``` ## Links - **Txt2Img-MHN:** [YonghaoXu/Txt2Img-MHN](https://github.com/YonghaoXu/Txt2Img-MHN) - **Taming Transformers:** [CompVis/taming-transformers](https://github.com/CompVis/taming-transformers) ## License MIT License - for academic use only.

Source context: 17 downloads · 0 likes · Pipeline image-to-image · Library diffusers · Repo BiliSakura/Txt2Img-MHN-VQGAN

Source context: 11 downloads · 0 likes · Pipeline image-to-image · Library diffusers · Repo BiliSakura/Txt2Img-MHN-VQGAN