arXiv:2410.15015cs.CV2024-10被引 36

用Mamba模型高效融合彩色与深度图像,提升显著目标检测精度。

MambaSOD: Dual Mamba-Driven Cross-Modal Fusion Network for RGB-D Salient Object Detection

  • 双Mamba结构分别提取RGB和深度图的长程依赖特征。
  • 跨模态Mamba融合模块实现两路特征互补信息的充分整合。
  • 在6个数据集上优于16个顶尖方法,计算开销低适合实际应用。

RGB-D显著目标检测旨在准确定位图像中最引人注目的区域。传统深度模型依赖卷积网络且忽略长程上下文,后续基于Transformer的方法虽缓解此问题但计算复杂度高。利用深度图的空间信息已被证明有效,核心挑战在于如何高效融合RGB与深度图的互补信息。本文提出双Mamba驱动的跨模态融合网络MambaSOD:首先使用双Mamba特征提取器对RGB和深度图建模多模态输入中的长程依赖,具有线性复杂度;随后设计跨模态融合Mamba,充分挖掘两路特征间的互补性。据我们所知,这是首次探索Mamba在RGB-D SOD任务中的潜力,提供了新视角。在六个主流数据集上的大量实验表明,该方法优于16个最先进的模型。源代码将发布于https://github.com/YueZhan721/MambaSOD。

原文摘要 · Abstract (English)

The purpose of RGB-D Salient Object Detection (SOD) is to pinpoint the most visually conspicuous areas within images accurately. While conventional deep models heavily rely on CNN extractors and overlook the long-range contextual dependencies, subsequent transformer-based models have addressed the issue to some extent but introduce high computational complexity. Moreover, incorporating spatial information from depth maps has been proven effective for this task. A primary challenge of this issue is how to fuse the complementary information from RGB and depth effectively. In this paper, we propose a dual Mamba-driven cross-modal fusion network for RGB-D SOD, named MambaSOD. Specifically, we first employ a dual Mamba-driven feature extractor for both RGB and depth to model the long-range dependencies in multiple modality inputs with linear complexity. Then, we design a cross-modal fusion Mamba for the captured multi-modal features to fully utilize the complementary information between the RGB and depth features. To the best of our knowledge, this work is the first attempt to explore the potential of the Mamba in the RGB-D SOD task, offering a novel perspective. Numerous experiments conducted on six prevailing datasets demonstrate our method's superiority over sixteen state-of-the-art RGB-D SOD models. The source code will be released at https://github.com/YueZhan721/MambaSOD.

RGB-D检测Mamba模型跨模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。