RIS FUSION

--- pipeline_tag: image-segmentation datasets: - ronniejiangC/MM-RIS arxiv: 2509.12710 tags: - referring-image-segmentation - image-fusion - multimodal --- # RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation This repository contains the model weights for **RIS-FUSION**, a cascaded framework presented in the paper [RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation](https://huggingface.co/papers/2509.12710). RIS-FUSION unifies text-driven infrared and visible image fusion with referring image segmentation (RIS) through joint optimization. The framework addresses the lack of goal-aligned supervision in existing methods by observing that RIS and text-driven fusion share a common objective: highlighting the object referred to by the text. At its core is the *LangGatedFusion* module, which injects textual features into the fusion backbone to enhance semantic alignment. ## Resources - **Paper**: [arXiv:2509.12710](https://huggingface.co/papers/2509.12710) - **GitHub Repository**: [SijuMa2003/RIS-FUSION](https://github.com/SijuMa2003/RIS-FUSION) - **Dataset (MM-RIS)**: [MM-RIS on Hugging Face](https://huggingface.co/datasets/ronniejiangC/MM-RIS) ## Sample Usage To evaluate the model using the official implementation, you can use the following command provided in the GitHub repository: ```bash python test.py \ --ckpt ./ckpts/risfusion/model_best_lavt.pth \ --test_parquet ./data/mm_ris_test.parquet \ --out_dir ./your_output_dir \ --bert_tokenizer ./bert/pretrained_weights/bert-base-uncased \ --ck_bert ./bert/pretrained_weights/bert-base-uncased ``` For detailed installation and training instructions, please refer to the [official GitHub repository](https://github.com/SijuMa2003/RIS-FUSION). ## Citation If you find this work useful, please consider citing the paper: ```bibtex @article{RIS-FUSION2025, title = {RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation}, author = {Ma, Siju and Gong, Changsiyu and Fan, Xiaofeng and Ma, Yong and Jiang, Chengjie}, journal = {arXiv preprint arXiv:2509.12710}, year = {2025} } ```