首个3D点云脉冲神经网络,效率高且能捕捉长距离依赖。
Efficient Spiking Point Mamba for Point Cloud Analysis
- 融合Mamba序列建模与脉冲网络时序特征提取机制。
- 在ScanObjectNN上提升整体准确率6.2%~7.4%,实例mIOU提高1.9%。
- 适合低功耗3D点云分析场景,尤其对能效敏感的应用。
生物启发的脉冲神经网络(SNN)为提取三维时空特征提供了高效节能的途径。然而,现有3D SNN难以处理长程依赖问题,直到最近Mamba的出现带来了更高的计算效率和序列建模能力。本文提出首个基于Mamba的3D SNN——Spiking Point Mamba(SPM)。由于直接将Mamba迁移至3D SNN表现不佳,SPM结合了Mamba的序列建模能力与SNN的时序特征提取优势。具体地,提出分层动态编码(HDE),引入动态时序机制以增强时间交互;设计脉冲Mamba模块(SMB),学习跨时间步特征并减少脉冲带来的信息损失;最后采用异构SNN-ANN架构进行基于脉冲的预训练与微调。相比以往最先进SNN模型,SPM在三个ScanObjectNN变体上分别提升总体准确率6.2%、6.1%和7.4%,在ShapeNetPart上提升实例mIOU 1.9%。同时,其能耗至少比对应ANN模型低3.5倍。代码将公开。
原文摘要 · Abstract (English)
Bio-inspired Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D spatio-temporal features. However, existing 3D SNNs have struggled with long-range dependencies until the recent emergence of Mamba, which offers superior computational efficiency and sequence modeling capability. In this work, we propose Spiking Point Mamba (SPM), the first Mamba-based SNN in the 3D domain. Due to the poor performance of simply transferring Mamba to 3D SNNs, SPM is designed to utilize both the sequence modeling capabilities of Mamba and the temporal feature extraction of SNNs. Specifically, we first introduce Hierarchical Dynamic Encoding (HDE), an improved direct encoding method that effectively introduces dynamic temporal mechanism, thereby facilitating temporal interactions. Then, we propose a Spiking Mamba Block (SMB), which builds upon Mamba while learning inter-time-step features and minimizing information loss caused by spikes. Finally, to further enhance model performance, we adopt an asymmetric SNN-ANN architecture for spike-based pre-training and finetune. Compared with the previous state-of-the-art SNN models, SPM improves OA by +6.2%, +6.1%, and +7.4% on three variants of ScanObjectNN, and boosts instance mIOU by +1.9% on ShapeNetPart. Meanwhile, its energy consumption is at least 3.5x lower than that of its ANN counterpart. The code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。