arXiv:2511.10046cs.CV2025-11被引 7

提出频域融合注意力机制,提升可见光与红外目标检测性能

FreDFT: Frequency Domain Fusion Transformer for Visible-Infrared Object Detection

  • 在频域构建跨模态注意力,挖掘可见光与红外特征互补性
  • 在多个公开数据集上超越现有方法,显著提升检测精度
  • 适合需要低光照/恶劣天气下精准检测的应用场景

可见光-红外目标检测因其在低光照、雾天和雨天等复杂环境下的优异表现而受到广泛关注。然而,由于不同传感器捕获的可见光与红外模态存在信息不平衡问题,导致跨模态融合不足,进而影响检测性能。现有方法大多在空间域使用Transformer捕捉互补特征,忽略了频域Transformer在挖掘互补信息方面的优势。为此,本文提出一种频域融合Transformer(FreDFT),设计了新型多模态频域注意力(MFDA)以挖掘模态间互补信息,并引入频域前馈层(FDFFL)结合多尺度频域特征融合策略,有效增强多模态特征。为缓解模态间信息不平衡,构建跨模态全局建模模块(CGMM),实现像素级的空间-通道联合跨模态特征交互。此外,设计局部特征增强模块(LFEM),通过多种卷积层与通道混洗操作强化局部特征表示,促进多模态融合。大量实验表明,FreDFT在多个公开数据集上优于当前最优方法。代码已开源:https://github.com/WenCongWu/FreDFT。

原文摘要 · Abstract (English)

Visible-infrared object detection has gained sufficient attention due to its detection performance in low light, fog, and rain conditions. However, visible and infrared modalities captured by different sensors exist the information imbalance problem in complex scenarios, which can cause inadequate cross-modal fusion, resulting in degraded detection performance. \textcolor{red}{Furthermore, most existing methods use transformers in the spatial domain to capture complementary features, ignoring the advantages of developing frequency domain transformers to mine complementary information.} To solve these weaknesses, we propose a frequency domain fusion transformer, called FreDFT, for visible-infrared object detection. The proposed approach employs a novel multimodal frequency domain attention (MFDA) to mine complementary information between modalities and a frequency domain feed-forward layer (FDFFL) via a mixed-scale frequency feature fusion strategy is designed to better enhance multimodal features. To eliminate the imbalance of multimodal information, a cross-modal global modeling module (CGMM) is constructed to perform pixel-wise inter-modal feature interaction in a spatial and channel manner. Moreover, a local feature enhancement module (LFEM) is developed to strengthen multimodal local feature representation and promote multimodal feature fusion by using various convolution layers and applying a channel shuffle. Extensive experimental results have verified that our proposed FreDFT achieves excellent performance on multiple public datasets compared with other state-of-the-art methods. The code of our FreDFT is linked at https://github.com/WenCongWu/FreDFT.

目标检测跨模态频域网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。