用脉冲神经网络高效处理点云,实现低功耗3D视觉识别。
Efficient 3D Recognition with Event-driven Spike Sparse Convolution
- 提出脉冲体素编码与稀疏卷积,将点云转为脉冲信号高效处理。
- 在ModelNet40上仅187万参数达91.7%准确率,优于现有脉冲网络。
- 首次实现事件驱动的统一3D视觉框架,适合类脑芯片部署。
脉冲神经网络(SNN)能以低功耗提取3D时空特征。点云是稀疏的3D空间数据,理论上非常适合SNN处理。然而,现有SNN在点云任务中表现有限,应用范围窄。我们将其归因于不当的预处理和特征提取方法。为此,我们提出脉冲体素编码(SVC),将3D点云映射为稀疏脉冲序列,降低存储开销并加速预处理。进一步设计脉冲稀疏卷积(SSC),高效提取3D稀疏点云特征。结合SVC与SSC,构建高效3D SNN主干网络E-3DSNN,兼容类脑硬件。例如,SSC可在类脑芯片上仅通过微调寻址函数实现。在ModelNet40、KITTI和Semantic KITTI数据集上的实验表明,E-3DSNN达到当前最优性能且效率显著。值得注意的是,其187万参数版本在ModelNet40上取得91.7%的顶1准确率,较现有最佳SNN基线(1430万参数)提升3.0个百分点。据我们所知,这是首个可直接训练、兼具事件驱动特性的3D SNN主干网络,能统一处理分类、检测与分割等多类3D视觉任务。代码已开源:https://github.com/bollossom/E-3DSNN/
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D spatio-temporal features. Point clouds are sparse 3D spatial data, which suggests that SNNs should be well-suited for processing them. However, when applying SNNs to point clouds, they often exhibit limited performance and fewer application scenarios. We attribute this to inappropriate preprocessing and feature extraction methods. To address this issue, we first introduce the Spike Voxel Coding (SVC) scheme, which encodes the 3D point clouds into a sparse spike train space, reducing the storage requirements and saving time on point cloud preprocessing. Then, we propose a Spike Sparse Convolution (SSC) model for efficiently extracting 3D sparse point cloud features. Combining SVC and SSC, we design an efficient 3D SNN backbone (E-3DSNN), which is friendly with neuromorphic hardware. For instance, SSC can be implemented on neuromorphic chips with only minor modifications to the addressing function of vanilla spike convolution. Experiments on ModelNet40, KITTI, and Semantic KITTI datasets demonstrate that E-3DSNN achieves state-of-the-art (SOTA) results with remarkable efficiency. Notably, our E-3DSNN (1.87M) obtained 91.7\% top-1 accuracy on ModelNet40, surpassing the current best SNN baselines (14.3M) by 3.0\%. To our best knowledge, it is the first direct training 3D SNN backbone that can simultaneously handle various 3D computer vision tasks (e.g., classification, detection, and segmentation) with an event-driven nature. Code is available: https://github.com/bollossom/E-3DSNN/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。