DEYOLO通过双增强机制融合可见光与红外图像,提升弱光环境目标检测性能。
DEYOLO: Dual-Feature-Enhancement YOLO for Cross-Modality Object Detection
- 设计双增强模块,分别优化语义与空间特征的跨模态融合
- 在M3FD和LLVIP数据集上显著超越现有最先进方法
- 适合需要多模态融合的目标检测场景,如夜间监控
弱光环境下目标检测困难,因RGB图像中物体不清晰。红外图像可提供补充的边缘信息,融合可见光与红外图像有望提升检测能力。然而,现有工作仅关注图像融合,未针对目标检测任务优化;且直接融合两类模态,忽略了它们之间的相互干扰。为此,提出面向跨模态目标检测的DEYOLO网络,通过语义-空间跨模态增强及新型双向解耦聚焦模块,实现RGB-IR的检测中心互增强。具体地,提出双语义增强通道权重分配模块(DECA)和双空间增强像素权重分配模块(DEPA),在特征空间聚合跨模态信息以提升表征能力,使融合服务于检测任务。同时,在DECA与DEPA中引入双增强机制,分别增强双模态融合与单模态表现,降低模态间干扰。此外,设计新型双向解耦聚焦模块,扩大骨干网络感受野,提升表示质量。在M3FD和LLVIP数据集上的大量实验表明,本方法显著优于当前最先进算法。代码已开源:https://github.com/chips96/DEYOLO。
原文摘要 · Abstract (English)
Object detection in poor-illumination environments is a challenging task as objects are usually not clearly visible in RGB images. As infrared images provide additional clear edge information that complements RGB images, fusing RGB and infrared images has potential to enhance the detection ability in poor-illumination environments. However, existing works involving both visible and infrared images only focus on image fusion, instead of object detection. Moreover, they directly fuse the two kinds of image modalities, which ignores the mutual interference between them. To fuse the two modalities to maximize the advantages of cross-modality, we design a dual-enhancement-based cross-modality object detection network DEYOLO, in which semantic-spatial cross modality and novel bi-directional decoupled focus modules are designed to achieve the detection-centered mutual enhancement of RGB-infrared (RGB-IR). Specifically, a dual semantic enhancing channel weight assignment module (DECA) and a dual spatial enhancing pixel weight assignment module (DEPA) are firstly proposed to aggregate cross-modality information in the feature space to improve the feature representation ability, such that feature fusion can aim at the object detection task. Meanwhile, a dual-enhancement mechanism, including enhancements for two-modality fusion and single modality, is designed in both DECAand DEPAto reduce interference between the two kinds of image modalities. Then, a novel bi-directional decoupled focus is developed to enlarge the receptive field of the backbone network in different directions, which improves the representation quality of DEYOLO. Extensive experiments on M3FD and LLVIP show that our approach outperforms SOTA object detection algorithms by a clear margin. Our code is available at https://github.com/chips96/DEYOLO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。