arXiv:2608.25367cs.CV2026-08

提出水下可见光与声呐多模态目标检测新方法,提升复杂环境识别能力。

RSFusionDet: Underwater RGB-Sonar Multimodal Object Detection

论文配图:RSFusionDet: Underwater RGB-Sonar Multimodal Object Detection
图 1 · 摘自论文原文
  • 设计跨模态注意力融合模块,解决可见光与声呐图像空间错位问题
  • 在RSFusion数据集上实现RGB/Sonar检测AP达76.4/48.6,匹配F1-score达83.4
  • 适合水下机器人、海洋探测等需多模态感知的场景

水下单模态目标检测面临成像挑战:光学图像受水中噪声和可视距离限制,声呐图像则缺乏物体结构信息。而光学图像富含结构信息,声呐图像受水下噪声影响小且可视距离更远,二者具有互补性。本文构建了首个RGB-声呐多模态目标检测数据集RSFusion,并提出评估指标。提出RSFusionDet模型,采用新的多模态检测结果表达方式,分析双模态特征,设计交叉注意力融合(CAFusion)模块处理空间错位特征,以及对象匹配头(OMHead)结合损失函数(OMLoss)实现跨模态对象匹配。在RSFusion数据集上,该方法达到76.4/48.6的检测平均精度(RGB/Sonar),对象匹配F1-score为83.4,优于其他检测模型。相比DINO基线,检测性能提升0.7/1.4 AP(RGB/Sonar),同时提供可靠的跨模态匹配。代码与数据集已开源。

原文摘要 · Abstract (English)

Underwater unimodal object detection faces many challenges in sensor imaging, such as optical images limited by underwater noise and visible distance, and sonar images limited by less object structural information. While, optical images have rich object structural information, and sonar images are less affected by underwater noise and have a longer visible distance. Optical (RGB modality) and sonar (Sonar modality) images have complementary information underwater. In this paper, we create an RGB-Sonar multimodal object detection dataset, \textbf{R}GB-\textbf{S}onar \textbf{Fusion} (RSFusion) and propose evaluation metrics for the benchmark. And we propose the \textbf{R}GB-\textbf{S}onar \textbf{Fusion} \textbf{Det}ector (RSFusionDet) with a new RGB-Sonar multimodal object detection result expression for RGB-Sonar multimodal object detection. We analyze the features of RGB and Sonar modal information, and design a Cross-Attention Fusion (CAFusion) module to fuse RGB-Sonar spatial misalignment features and Object Matching Head (OMHead) with Loss (OMLoss) to match identical objects in RGB-Sonar modalities. Our RSFusionDet achieves 76.4/48.6 AP (RGB/Sonar) for object detection and 83.4 \(\text{F1-Score}_{match}\) for object matching, on RSFusion, which outperforms other object detection models. Compared with the DINO baseline, our method improves by 0.7/1.4 AP (RGB/Sonar) while simultaneously providing reliable cross-modal object matching. The code and datasets are publicly available at https://github.com/LEFTeyex/RSFusionDet.

多模态检测水下感知目标匹配声呐融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。