arXiv:2501.01648cs.CVcs.MM2025-01被引 2

提出双模态互学习网络,提升RGB-D显著目标检测精度

Dual Mutual Learning Network with Global-local Awareness for RGB-D Salient Object Detection

  • 设计空间与通道级双互融合模块,自适应对齐多模态特征
  • 在6个数据集上超越24种方法,平均指标提升约3%
  • 适合关注多模态融合与图像显著性检测的研究者

RGB-D显著目标检测(SOD)旨在通过联合建模彩色图像与深度信息,突出场景中的显著区域,是具有挑战性的像素级预测任务。近期双注意力机制被用于增强检测性能,但多数方法采用人工强制的融合范式,直接融合跨模态特征而未考虑RGB与深度之间的本质差异,可能导致性能下降。此外,全局与局部信息带来的长距离依赖关系使统一高效的融合策略难以实现。为此,本文提出一种带有全局-局部感知能力的双互学习网络GL-DMNet。具体地,设计位置互融合模块与通道互融合模块,分别在空间和通道维度挖掘不同模态间的相互依赖关系;同时采用基于级联Transformer增强重建的高效解码器,融合多层级特征。在六个基准数据集上的大量实验表明,所提GL-DMNet优于24种现有方法,在四个评估指标上相较第二佳模型S3Net平均提升约3%。代码与结果见https://github.com/kingkung2016/GL-DMNet。

原文摘要 · Abstract (English)

RGB-D salient object detection (SOD), aiming to highlight prominent regions of a given scene by jointly modeling RGB and depth information, is one of the challenging pixel-level prediction tasks. Recently, the dual-attention mechanism has been devoted to this area due to its ability to strengthen the detection process. However, most existing methods directly fuse attentional cross-modality features under a manual-mandatory fusion paradigm without considering the inherent discrepancy between the RGB and depth, which may lead to a reduction in performance. Moreover, the long-range dependencies derived from global and local information make it difficult to leverage a unified efficient fusion strategy. Hence, in this paper, we propose the GL-DMNet, a novel dual mutual learning network with global-local awareness. Specifically, we present a position mutual fusion module and a channel mutual fusion module to exploit the interdependencies among different modalities in spatial and channel dimensions. Besides, we adopt an efficient decoder based on cascade transformer-infused reconstruction to integrate multi-level fusion features jointly. Extensive experiments on six benchmark datasets demonstrate that our proposed GL-DMNet performs better than 24 RGB-D SOD methods, achieving an average improvement of ~3% across four evaluation metrics compared to the second-best model (S3Net). Codes and results are available at https://github.com/kingkung2016/GL-DMNet.

RGB-D检测多模态融合显著性分割Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。