用小波变换融合可见光与红外图像，提升目标检测精度
WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection
- 通过小波分解提取多频段特征，用Mamba结构融合低频信息
- 在四个数据集上平均检测准确率提升4.5%，优于现有方法
- 适合做跨模态目标检测的工程师和研究人员参考
利用可见光（RGB）与红外（IR）图像的互补特性，可显著提升目标检测性能。本文提出WaveMamba，一种基于离散小波变换（DWT）分解并融合两者独特频率特征的跨模态融合方法。设计了结合逆离散小波变换（IDWT）的改进检测头，减少信息损失，生成最终检测结果。核心是波浪形马尔可夫融合模块（WMFB），实现低/高频子带的全面融合。其中，基于Mamba框架的低频融合块（LMFB）先通过通道交换进行初步融合，再用先进门控注意力机制深度整合；高频特征则采用“绝对最大”融合策略增强。实验表明，该方法在四个基准测试上均超越现有最优方案，平均mAP提升4.5%。
原文摘要 · Abstract (English)
Leveraging the complementary characteristics of visible (RGB) and infrared (IR) imagery offers significant potential for improving object detection. In this paper, we propose WaveMamba, a cross-modality fusion method that efficiently integrates the unique and complementary frequency features of RGB and IR decomposed by Discrete Wavelet Transform (DWT). An improved detection head incorporating the Inverse Discrete Wavelet Transform (IDWT) is also proposed to reduce information loss and produce the final detection results. The core of our approach is the introduction of WaveMamba Fusion Block (WMFB), which facilitates comprehensive fusion across low-/high-frequency sub-bands. Within WMFB, the Low-frequency Mamba Fusion Block (LMFB), built upon the Mamba framework, first performs initial low-frequency feature fusion with channel swapping, followed by deep fusion with an advanced gated attention mechanism for enhanced integration. High-frequency features are enhanced using a strategy that applies an ``absolute maximum" fusion approach. These advancements lead to significant performance gains, with our method surpassing state-of-the-art approaches and achieving average mAP improvements of 4.5% on four benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。