提出红外主导的融合方法，提升多光谱目标检测性能
Multispectral Detection Transformer with Infrared-Centric Feature Fusion
- 以红外图像为主导，设计轻量级跨模态融合模块
- 在FLIR和LLVIP数据集上显著优于现有方法，精度提升3.2%以上
- 适合需要夜间或复杂环境检测的应用场景
多光谱目标检测旨在利用可见光（RGB）与红外（IR）模态的互补信息，实现多样环境下的鲁棒检测。通过小波分析和实证观察发现，红外图像包含对目标检测至关重要的高频结构信息，因此采用红外主导策略更具优势。为此，我们提出红外主导融合（IC-Fusion）方法，该方法轻量化且具备模态感知能力，优先保留红外特征并有效融合可见光语义信息。IC-Fusion采用紧凑的可见光骨干网络，并设计新型融合模块：多尺度特征蒸馏（MSFD）块用于增强可见光特征，三阶段融合结构包含跨模态通道洗牌门（CCSG）、跨模态大核门（CLKG）和通道洗牌投影（CSP），促进高效跨模态交互。在FLIR与LLVIP基准测试中，所提方法表现出卓越的有效性与效率，验证了红外主导融合的优势。代码已开源：https://github.com/smin-hwang/IC-Fusion。
原文摘要 · Abstract (English)
Multispectral object detection aims to leverage complementary information from visible (RGB) and infrared (IR) modalities to enable robust performance under diverse environmental conditions. Our key insight, derived from wavelet analysis and empirical observations, is that IR images contain structurally rich high-frequency information critical for object detection, making an infrared-centric approach highly effective. To capitalize on this finding, we propose Infrared-Centric Fusion (IC-Fusion), a lightweight and modality-aware sensor fusion method that prioritizes infrared features while effectively integrating complementary RGB semantic context. IC-Fusion adopts a compact RGB backbone and designs a novel fusion module comprising a Multi-Scale Feature Distillation (MSFD) block to enhance RGB features and a three-stage fusion block with a Cross-Modal Channel Shuffle Gate (CCSG), a Cross-Modal Large Kernel Gate (CLKG), and a Channel Shuffle Projection (CSP) to facilitate effective cross-modal interaction. Experiments on the FLIR and LLVIP benchmarks demonstrate the superior effectiveness and efficiency of our IR-centric fusion strategy, further validating its benefits. Our code is available at https://github.com/smin-hwang/IC-Fusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。