用新方法将点云转为序列,让Mamba模型高效处理3D数据。
NIMBA: Towards Robust and Principled Processing of Point Clouds With SSMs
- 不复制数据,直接保留3D结构生成1维序列
- 在ModelNet40和ScanObjectNN上达顶尖性能
- 无需位置编码,速度快于基于Transformer的模型
Transformers在文本、2D和3D视觉等任务中占据主导地位,但其注意力机制的二次复杂度限制了在高分辨率3D数据(如点云)中的效率。最近,状态空间模型(SSMs)如Mamba因其线性复杂度、可扩展性和长序列任务中的高性能成为有前景的替代方案。然而,将SSMs应用于点云时的关键挑战在于:点云本身无序,而Mamba等循环模型依赖方向性顺序处理。此前研究通过沿多个方向重排点云并拼接生成单一1维序列来解决。本文提出一种新方法,将点云转换为保持3D空间结构的1维序列,无需数据复制,使Mamba能以近乎排列不变的方式有效处理。相比现有方法,本方法无需位置编码,序列更短,同时在ModelNet40和ScanObjectNN数据集上达到当前最优结果,并在准确率和效率上超越基于Transformer的模型。
原文摘要 · Abstract (English)
Transformers have become dominant in large-scale deep learning tasks across various domains, including text, 2D and 3D vision. However, the quadratic complexity of their attention mechanism limits their efficiency as the sequence length increases, particularly in high-resolution 3D data such as point clouds. Recently, state space models (SSMs) like Mamba have emerged as promising alternatives, offering linear complexity, scalability, and high performance in long-sequence tasks. The key challenge in the application of SSMs in this domain lies in reconciling the non-sequential structure of point clouds with the inherently directional (or bi-directional) order-dependent processing of recurrent models like Mamba. To achieve this, previous research proposed reorganizing point clouds along multiple directions or predetermined paths in 3D space, concatenating the results to produce a single 1D sequence capturing different views. In our work, we introduce a method to convert point clouds into 1D sequences that maintain 3D spatial structure with no need for data replication, allowing Mamba sequential processing to be applied effectively in an almost permutation-invariant manner. In contrast to other works, we found that our method does not require positional embeddings and allows for shorter sequence lengths while still achieving state-of-the-art results in ModelNet40 and ScanObjectNN datasets and surpassing Transformer-based models in both accuracy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。