轻量级神经架构搜索,让脉冲神经网络又快又准
LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks
- 基于无训练剪枝的搜索机制,自动找最优脉冲网络结构
- 在CIFAR10/100上达最新水平,在DVS128-Gesture上提升4.49%
- 搜索速度比前人快98倍,适合边缘设备部署
脉冲神经网络(SNNs)因其能耗低、激活稀疏和适合边缘实时处理而备受关注。但现有SNN方法多沿用传统人工神经网络架构,导致性能不理想。本文提出LightSNN,一种专为SNN设计的快速高效神经网络架构搜索(NAS)方法,能自动寻找兼顾准确率与效率的稀疏结构。基于SNASNet框架,采用含反向连接的单元化搜索空间,构建无训练剪枝的NAS机制。通过稀疏感知的汉明距离评估不同数据样本的脉冲激活模式,实现高效筛选。在静态数据集CIFAR10、CIFAR100和脉冲数据集DVS128-Gesture上进行充分实验。LightSNN在CIFAR10和CIFAR100上达到当前最佳表现,在DVS128-Gesture上性能提升4.49%,搜索时间相比SNASNet提速98倍,且在该数据集上比最优现有方法快30%。代码已开源。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) are highly regarded for their energy efficiency, inherent activation sparsity, and suitability for real-time processing in edge devices. However, most current SNN methods adopt architectures resembling traditional artificial neural networks (ANNs), leading to suboptimal performance when applied to SNNs. While SNNs excel in energy efficiency, they have been associated with lower accuracy levels than traditional ANNs when utilizing conventional architectures. In response, in this work we present LightSNN, a rapid and efficient Neural Network Architecture Search (NAS) technique specifically tailored for SNNs that autonomously leverages the most suitable architecture, striking a good balance between accuracy and efficiency by enforcing sparsity. Based on the spiking NAS network (SNASNet) framework, a cell-based search space including backward connections is utilized to build our training-free pruning-based NAS mechanism. Our technique assesses diverse spike activation patterns across different data samples using a sparsity-aware Hamming distance fitness evaluation. Thorough experiments are conducted on both static (CIFAR10 and CIFAR100) and neuromorphic datasets (DVS128-Gesture). Our LightSNN model achieves state-of-the-art results on CIFAR10 and CIFAR100, improves performance on DVS128Gesture by 4.49\%, and significantly reduces search time most notably offering a $98\times$ speedup over SNASNet and running 30\% faster than the best existing method on DVS128Gesture. Code is available on Github at: https://github.com/YesmineAbdennadher/LightSNN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。