arXiv:2410.21211cs.CV2024-10被引 1

用线性复杂度的Mamba改进点云分割,速度更快且更省内存。

Exploring contextual modeling with linear complexity for point cloud segmentation

  • 改造Mamba结构,加入双向扫描和去因果卷积以适配点云特性。
  • 在多个基准上提升0.8 mIoU,推理速度加快42.1%,内存效率提高5.53倍。
  • 适合需要高效点云分割的工业应用或实时系统开发者。

点云分割是3D理解的重要任务,传统方法依赖CNN或Transformer。近期,Mamba因其线性复杂度的长程上下文建模能力成为有前景的替代方案,但早期尝试均未超越最优的CNN与Transformer方法。本文通过识别高效点云分割架构的关键组件,发现空间局部性和鲁棒的上下文理解对性能至关重要。同时,我们指出原始Mamba的因果卷积不适用于无序点云,且单向扫描策略引入方向偏见。为此,我们移除因果卷积,引入新型分段双向SSM,显著增强空间关系捕捉能力。最终提出MEEPO架构,融合CNN与Mamba优势,在多个主流数据集上较PTv3提升最高0.8 mIoU,推理速度提升42.1%,内存效率提高5.53倍。

原文摘要 · Abstract (English)

Point cloud segmentation is an important topic in 3D understanding that has traditionally has been tackled using either the CNN or Transformer. Recently, Mamba has emerged as a promising alternative, offering efficient long-range contextual modeling capabilities without the quadratic complexity associated with Transformer's attention mechanisms. However, despite Mamba's potential, early efforts have all failed to achieve better performance than the best CNN-based and Transformer-based methods. In this work, we address this challenge by identifying the key components of an effective and efficient point cloud segmentation architecture. Specifically, we show that: 1) Spatial locality and robust contextual understanding are critical for strong performance, and 2) Mamba features linear computational complexity, offering superior data and inference efficiency compared to Transformers, while still being capable of delivering strong contextual understanding. Additionally, we further enhance the standard Mamba specifically for point cloud segmentation by identifying its two key shortcomings. First, the enforced causality in the original Mamba is unsuitable for processing point clouds that have no such dependencies. Second, its unidirectional scanning strategy imposes a directional bias, hampering its ability to capture the full context of unordered point clouds in a single pass. To address these issues, we carefully remove the causal convolutions and introduce a novel Strided Bidirectional SSM to enhance the model's capability to capture spatial relationships. Our efforts culminate in the development of a novel architecture named MEEPO, which effectively integrates the strengths of CNN and Mamba. MEEPO surpasses the previous state-of-the-art method, PTv3, by up to +0.8 mIoU on multiple key benchmark datasets, while being 42.1% faster and 5.53x more memory efficient.

点云分割Mamba线性复杂度高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。