arXiv:2607.08076cs.CVcs.AI2026-07被引 1

提出LDFE模块,提升双流RGB-IR目标检测在极端条件下的性能。

LDFE: Laplacian Decoupled Feature Enhancement Block for Dual-Stream CNN-based RGB-IR Object Detection

论文配图:LDFE: Laplacian Decoupled Feature Enhancement Block for Dual-Stream CNN-based RGB-IR Object Detection
图 1 · 摘自论文原文
  • 通过拉普拉斯金字塔分解特征,分阶段进行全局局部融合与去噪。
  • 在多个数据集上mAP超越当前最优方法2.0%~6.2%。
  • 适合需要跨模态融合的红外与可见光目标检测场景。

RGB与红外图像间的互补信息可显著提升极端条件下的目标检测性能。现有方法多采用基于YOLO的双流卷积神经网络骨干网络,并聚焦于特征融合设计。本文提出拉普拉斯解耦特征增强模块(LDFE),用于融合双流CNN骨干网络不同阶段的特征。LDFE通过全局-局部分解、去噪、融合与重构的顺序设计,同时考虑模态特性和结构特性。首先基于拉普拉斯金字塔将特征分离为全局与局部成分,再分别通过全局状态空间增强模块(GS2E)和局部卷积相关增强模块(LC2E)进行去噪与融合。GS2E采用双分支结构,利用辅助模态的跨模态注意力动态抑制主模态噪声,并通过状态空间模型捕捉主模态全局特征中的长程依赖。为实现双向交互,两模态系统性交替主/辅角色。LC2E则抑制局部特征噪声,结合空间与通道维度及三重卷积提取细粒度细节以供融合。该设计在M3FD、DroneVehicle、LLVIP、FLIR-Aligned、KAIST和VEDAI数据集上分别取得比SOTA方法高6.2%、3.7%、4.7%、2.3%、4.1%和2.0%的mAP,显著提升性能。

原文摘要 · Abstract (English)

The complementary information between RGB and IR images can significantly enhance object detection performance under extreme conditions. Existing methods prefer dual-stream CNN backbones built upon YOLO for feature extraction and focus on the design of feature fusion. In this paper, we introduce the Laplacian Decoupled Feature Enhancement block (LDFE) to fuse features from different stages of the dual-stream CNN backbone. By design, LDFE simultaneously considers the characteristics of modalities and structures for feature fusion by employing global-local decomposition, denoising, fusion, and reconstruction, sequentially. The LDFE first separates features into global and local components based on Laplacian Pyramid, and then performs denoising and fusion based on Global State Space Enhancement module (GS2E) and Local Convolutional Correlation Enhancement module (LC2E) separately. Specifically, the GS2E conducts a two-branch architecture for the main and auxiliary modalities. It dynamically suppresses noise in the main modality through cross-modal attention derived from the auxiliary modality, while employing a State Space Model to capture long-range dependencies within the global feature representations of the main modality. To obtain bidirectional interaction, the two modalities systematically alternate their main/auxiliary roles. Moreover, the LC2E suppresses noise in local features and leverages spatial and channel dimension along with triple convolution to extract fine-grained details for fusion. These innovative designs achieve a significant performance improvement, with mAP surpassing the SOTA methods 6.2%, 3.7%, 4.7%, 2.3%, 4.1% and 2.0% on M3FD, DroneVehicle, LLVIP, FLIR-Aligned, KAIST and VEDAI datasets,respectively.

目标检测双流网络红外可见光特征融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。