arXiv:2412.09258cs.CV2024-12中稿 · AAAI被引 38

通过频率分解提升红外可见光目标检测,显著改善复杂环境下的识别效果。

FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection

论文配图:FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection
图 1 · 摘自论文原文
  • 基于频域特性分离高/低频特征,分别用DCT和动态感受野建模
  • 在LLVIP、FLIR、M3FD上分别达到96.2%、82.9%、83.5% mAP
  • 适合需要跨模态融合的红外视觉检测场景

红外-可见光目标检测(IVOD)旨在利用红外与可见光图像的互补信息,从而提升复杂环境下的检测性能。然而,现有方法常忽略互补信息的频率特性,如可见光中丰富的高频细节和红外中的低频热信息,制约了检测效果。为此,本文提出一种新型频率驱动特征分解网络FD2-Net,有效捕捉多模态视觉空间中互补信息的独特频率表征。具体而言,设计特征分解编码器:高频单元(HFU)采用离散余弦变换(DCT)提取代表性高频特征;低频单元(LFU)使用动态感受野建模多样化目标的多尺度上下文。进一步提出无参数的互补优势策略,实现跨频域无缝融合。创新性地引入多模态重建机制,恢复特征提取过程丢失的图像细节,充分挖掘红外与可见光间的互补信息以增强整体表征能力。大量实验表明,FD2-Net在多个主流IVOD基准上超越现有最优模型:在LLVIP上达96.2% mAP,FLIR上达82.9% mAP,M3FD上达83.5% mAP。

原文摘要 · Abstract (English)

Infrared-visible object detection (IVOD) seeks to harness the complementary information in infrared and visible images, thereby enhancing the performance of detectors in complex environments. However, existing methods often neglect the frequency characteristics of complementary information, such as the abundant high-frequency details in visible images and the valuable low-frequency thermal information in infrared images, thus constraining detection performance. To solve this problem, we introduce a novel Frequency-Driven Feature Decomposition Network for IVOD, called FD2-Net, which effectively captures the unique frequency representations of complementary information across multimodal visual spaces. Specifically, we propose a feature decomposition encoder, wherein the high-frequency unit (HFU) utilizes discrete cosine transform to capture representative high-frequency features, while the low-frequency unit (LFU) employs dynamic receptive fields to model the multi-scale context of diverse objects. Next, we adopt a parameter-free complementary strengths strategy to enhance multimodal features through seamless inter-frequency recoupling. Furthermore, we innovatively design a multimodal reconstruction mechanism that recovers image details lost during feature extraction, further leveraging the complementary information from infrared and visible images to enhance overall representational capacity. Extensive experiments demonstrate that FD2-Net outperforms state-of-the-art (SOTA) models across various IVOD benchmarks, i.e. LLVIP (96.2% mAP), FLIR (82.9% mAP), and M3FD (83.5% mAP).

红外检测跨模态频域分析目标检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。