Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
Official model weights for the paper: "Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting" Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg...
Model source
Source description
Official model weights for the paper:
"Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting"
Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg Maier, Thomas Längle, Markus Klute
Sources
1 sourceVerified Sep 8
Model artifacts
13 artifactsSource excerpts
2 excerptsWe present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding spatial pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention.
Please refer to the detailed instructions and example notebooks (Preprocess, Evaluate, and Training) available on the official GitHub repository.
| File | mIoU ↑ | Img./s ↑ |
|---|---|---|
BCAF_SpectralWaste_rgb1024_hsi5_best.pth | 76.4 ± 0.4% | 31 |
BCAF_SpectralWaste_rgb512_hsi5_best.pth | 75.4 ± 0.2% | 55 |
BCAF_SpectralWaste_rgb256_hsi5_best.pth | 71.1 ± 0.4% | 54 |
logitfusion_SpectralWaste_rgb1024_hsi5_best.pth | 72.6 ± 0.8% | 39 |
| File | mIoU ↑ | Img./s ↑ |
|---|---|---|
swin_t_SpectralWaste_rgb_256_best.pth | 65.8 ± 1.2% | 141 |
swin_t_SpectralWaste_rgb_512_best.pth | 71.1 ± 0.6% | 135 |
swin_t_SpectralWaste_rgb_1024_best.pth | 71.6 ± 0.3% | 60 |
swin_t_SpectralWaste_rgb_2048_best.pth | 68.4 ± 0.8% | 15 |
| File | mIoU ↑ | Img./s ↑ |
|---|---|---|
swin_t_SpectralWaste_hsi_1_best.pth | 60.9 ± 0.2% | 141 |
adapted_swin_t_SpectralWaste_hsi_3_best.pth | 59.7 ± 0.7% | 114 |
adapted_swin_t_SpectralWaste_hsi_5_best.pth | 60.3 ± 0.9% | 119 |
adapted_swin_t_SpectralWaste_hsi_7_best.pth | 59.0 ± 1.5% | 91 |
adapted_swin_t_SpectralWaste_hsi_10_best.pth | 57.8 ± 1.2% | 68 |
@article{funk2026bcaf,
title={Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI
for Multimodal Automated Waste Sorting},
author={Jonas V. Funk and Lukas Roming and Andreas Michel and Paul B{\"{a}}cker
and Georg Maier and Thomas L{\"{a}}ngle and Markus Klute},
year={2026},
eprint={2603.13941},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2603.13941}
}
adapted_swin_t_SpectralWaste_hsi_5_best.pth
pth · 174 MB · SHA-256 99a1440b1512…0c93 · Hugging Face
Downloadadapted_swin_t_SpectralWaste_hsi_7_best.pth
pth · 174 MB · SHA-256 8247d68a7d48…76a3 · Hugging Face
DownloadBCAF_SpectralWaste_rgb1024_hsi5_best.pth
pth · 387 MB · SHA-256 6f0894ad616d…2e0e · Hugging Face
DownloadBCAF_SpectralWaste_rgb256_hsi5_best.pth
pth · 387 MB · SHA-256 e3fb7b169297…9a30 · Hugging Face
DownloadBCAF_SpectralWaste_rgb512_hsi5_best.pth
pth · 387 MB · SHA-256 226715f47c1a…e9e2 · Hugging Face
Downloadlogitfusion_SpectralWaste_rgb1024_hsi5_best.pth
pth · 297 MB · SHA-256 e5d142bc1d89…e8c8 · Hugging Face
Downloadswin_t_SpectralWaste_rgb_1024_best.pth
pth · 123 MB · SHA-256 6d503bb07f23…f3a6 · Hugging Face
Downloadswin_t_SpectralWaste_rgb_2048_best.pth
pth · 123 MB · SHA-256 c40a5061a690…c1a3 · Hugging Face
Downloadswin_t_SpectralWaste_rgb_256_best.pth
pth · 123 MB · SHA-256 00f8b05f9c63…574d · Hugging Face
Downloadswin_t_SpectralWaste_rgb_512_best.pth
pth · 123 MB · SHA-256 d403c5a2afc6…d6c8 · Hugging Face
Download--- datasets: - spectralwaste license: mit metrics: - mean_iou pipeline_tag: image-segmentation tags: - semantic-segmentation - computer-vision - multimodal - hyperspectral-imaging - waste-sorting - swin-transformer - pytorch model-index: - name: BCAF_SpectralWaste_rgb1024_hsi5 results: - task: type: image-segmentation dataset: name: SpectralWaste type: spectralwaste metrics: - type: mean_iou value: 76.4 --- # BCAF — Bidirectional Cross-Attention Fusion Official model weights for the paper: **"Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting"** Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg Maier, Thomas Längle, Markus Klute [](https://arxiv.org/abs/2603.13941) [](https://github.com/jonasvilhofunk/BCAF_2026) ## Abstract We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding spatial pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. --- ## Usage Please refer to the detailed instructions and example notebooks (Preprocess, Evaluate, and Training) available on the [official GitHub repository](https://github.com/jonasvilhofunk/BCAF_2026). --- ## Checkpoints ### Fusion Models | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `BCAF_SpectralWaste_rgb1024_hsi5_best.pth` | **76.4 ± 0.4%** | 31 | | `BCAF_SpectralWaste_rgb512_hsi5_best.pth` | 75.4 ± 0.2% | 55 | | `BCAF_SpectralWaste_rgb256_hsi5_best.pth` | 71.1 ± 0.4% | 54 | | `logitfusion_SpectralWaste_rgb1024_hsi5_best.pth` | 72.6 ± 0.8% | 39 | ### Unimodal RGB | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `swin_t_SpectralWaste_rgb_256_best.pth` | 65.8 ± 1.2% | 141 | | `swin_t_SpectralWaste_rgb_512_best.pth` | 71.1 ± 0.6% | 135 | | `swin_t_SpectralWaste_rgb_1024_best.pth` | 71.6 ± 0.3% | 60 | | `swin_t_SpectralWaste_rgb_2048_best.pth` | 68.4 ± 0.8% | 15 | ### Unimodal HSI | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `swin_t_SpectralWaste_hsi_1_best.pth` | 60.9 ± 0.2% | 141 |...
Source context: 0 downloads · 0 likes · Pipeline image-segmentation · Repo jonasvilhofunk/BCAF_2026