BCAF 2026

--- datasets: - spectralwaste license: mit metrics: - mean_iou pipeline_tag: image-segmentation tags: - semantic-segmentation - computer-vision - multimodal - hyperspectral-imaging - waste-sorting - swin-transformer - pytorch model-index: - name: BCAF_SpectralWaste_rgb1024_hsi5 results: - task: type: image-segmentation dataset: name: SpectralWaste type: spectralwaste metrics: - type: mean_iou value: 76.4 --- # BCAF — Bidirectional Cross-Attention Fusion Official model weights for the paper: **"Bidirectional Cross-Attention Fusion of High-Res RGB and Low-Res HSI for Multimodal Automated Waste Sorting"** Jonas V. Funk, Lukas Roming, Andreas Michel, Paul Bäcker, Georg Maier, Thomas Längle, Markus Klute [](https://arxiv.org/abs/2603.13941)  [](https://github.com/jonasvilhofunk/BCAF_2026) ## Abstract We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding spatial pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. --- ## Usage Please refer to the detailed instructions and example notebooks (Preprocess, Evaluate, and Training) available on the [official GitHub repository](https://github.com/jonasvilhofunk/BCAF_2026). --- ## Checkpoints ### Fusion Models | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `BCAF_SpectralWaste_rgb1024_hsi5_best.pth` | **76.4 ± 0.4%** | 31 | | `BCAF_SpectralWaste_rgb512_hsi5_best.pth` | 75.4 ± 0.2% | 55 | | `BCAF_SpectralWaste_rgb256_hsi5_best.pth` | 71.1 ± 0.4% | 54 | | `logitfusion_SpectralWaste_rgb1024_hsi5_best.pth` | 72.6 ± 0.8% | 39 | ### Unimodal RGB | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `swin_t_SpectralWaste_rgb_256_best.pth` | 65.8 ± 1.2% | 141 | | `swin_t_SpectralWaste_rgb_512_best.pth` | 71.1 ± 0.6% | 135 | | `swin_t_SpectralWaste_rgb_1024_best.pth` | 71.6 ± 0.3% | 60 | | `swin_t_SpectralWaste_rgb_2048_best.pth` | 68.4 ± 0.8% | 15 | ### Unimodal HSI | File | mIoU ↑ | Img./s ↑ | |---|---|---| | `swin_t_SpectralWaste_hsi_1_best.pth` | 60.9 ± 0.2% | 141 |...