融合Transformer与Mamba，提升3D语义分割的效率与精度
HybridTM: Combining Transformer and Mamba for 3D Semantic Segmentation
- 将Transformer与Mamba在细粒度层级结合，兼顾长程依赖与局部特征
- 在ScanNet、ScanNet200和nuScenes上达到最新最佳性能
- 线性复杂度下实现高效建模，适合大规模点云处理
基于Transformer的方法凭借强大的注意力机制在3D语义分割中表现卓越，但其二次复杂度限制了对大规模点云中长程依赖的建模。近期基于Mamba的方法虽以线性复杂度实现高效处理，但在提取3D特征时存在表征能力不足的问题。如何有效结合两者优势仍是该领域未解挑战。本文提出HybridTM，首个融合Transformer与Mamba的3D语义分割混合架构，并引入内层混合策略（Inner Layer Hybrid Strategy），在更细粒度上同时整合注意力机制与Mamba结构，实现长程依赖与细粒度局部特征的协同捕获。大量实验表明，HybridTM在多种室内与室外数据集上具备优异有效性与泛化能力。尤其在ScanNet、ScanNet200和nuScenes基准上达到当前最优性能。代码将开源于https://github.com/deepinact/HybridTM。
原文摘要 · Abstract (English)
Transformer-based methods have demonstrated remarkable capabilities in 3D semantic segmentation through their powerful attention mechanisms, but the quadratic complexity limits their modeling of long-range dependencies in large-scale point clouds. While recent Mamba-based approaches offer efficient processing with linear complexity, they struggle with feature representation when extracting 3D features. However, effectively combining these complementary strengths remains an open challenge in this field. In this paper, we propose HybridTM, the first hybrid architecture that integrates Transformer and Mamba for 3D semantic segmentation. In addition, we propose the Inner Layer Hybrid Strategy, which combines attention and Mamba at a finer granularity, enabling simultaneous capture of long-range dependencies and fine-grained local features. Extensive experiments demonstrate the effectiveness and generalization of our HybridTM on diverse indoor and outdoor datasets. Furthermore, our HybridTM achieves state-of-the-art performance on ScanNet, ScanNet200, and nuScenes benchmarks. The code will be made available at https://github.com/deepinact/HybridTM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。