融合短波与长波红外图像,一步完成船只检测与图像融合。
LSFDNet: A Single-Stage Fusion and Detection Network for Ships Using SWIR and LWIR
- 设计单阶段融合检测网络,通过特征交互提升性能。
- 在两个数据集上实现更高检测精度,融合图像更清晰。
- 适合遥感、海上监测等复杂环境下的目标识别任务。
传统船只检测方法多依赖可见光或单一红外图像,在光照变化和浓雾等复杂场景下表现受限。为此,本文探索短波红外(SWIR)与长波红外(LWIR)在船只检测中的优势,提出一种新型单阶段图像融合与检测算法LSFDNet。该算法通过融合与检测子网之间的特征交互,显著提升检测性能并生成视觉效果优良的融合图像。为增强融合图像中目标的显著性并优化下游检测任务,引入多层级交叉融合(MLCF)模块,结合检测任务中的敏感特征,跨模态、多尺度、多任务聚合语义丰富的融合特征。同时,在对象增强(OE)损失函数中引入检测任务的位置先验,进一步保留融合图像中的目标语义。检测任务也利用融合任务的初步融合特征,补充原始SWIR与LWIR特征,从而提升检测能力。此外,构建了近岸船只长短波注册数据集NSLSR,填补该领域空白。在两个数据集上的验证表明,所提算法具有明显优势。源代码与数据集已开源。
原文摘要 · Abstract (English)
Traditional ship detection methods primarily rely on single-modal approaches, such as visible or infrared images, which limit their application in complex scenarios involving varying lighting conditions and heavy fog. To address this issue, we explore the advantages of short-wave infrared (SWIR) and long-wave infrared (LWIR) in ship detection and propose a novel single-stage image fusion detection algorithm called LSFDNet. This algorithm leverages feature interaction between the image fusion and object detection subtask networks, achieving remarkable detection performance and generating visually impressive fused images. To further improve the saliency of objects in the fused images and improve the performance of the downstream detection task, we introduce the Multi-Level Cross-Fusion (MLCF) module. This module combines object-sensitive fused features from the detection task and aggregates features across multiple modalities, scales, and tasks to obtain more semantically rich fused features. Moreover, we utilize the position prior from the detection task in the Object Enhancement (OE) loss function, further increasing the retention of object semantics in the fused images. The detection task also utilizes preliminary fused features from the fusion task to complement SWIR and LWIR features, thereby enhancing detection performance. Additionally, we have established a Nearshore Ship Long-Short Wave Registration (NSLSR) dataset to train effective SWIR and LWIR image fusion and detection networks, bridging a gap in this field. We validated the superiority of our proposed single-stage fusion detection algorithm on two datasets. The source code and dataset are available at https://github.com/Yanyin-Guo/LSFDNet
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。