一次性压缩脉冲神经网络,能效更高且精度提升明显。
Spiking Brain Compression: Post-Training Second-order Compression for Spiking Neural Networks
- 用脉冲流目标替代传统方法,单次反向传播完成压缩与参数重标定。
- 在图像数据集上实现比人工神经网络压缩方案高1到2位数的准确率提升。
- 适合资源受限场景下快速部署大型脉冲神经网络,尤其对小样本校准鲁棒。
脉冲神经网络(SNNs)作为新一代低功耗神经网络,适用于类脑硬件部署。由于类脑硬件内存与计算资源有限,参数剪枝与量化被用于提升SNN效率。现有先进方法需多次压缩与训练迭代,对预训练或超大SNN成本高昂。本文提出一种新型一次性后训练压缩框架Spiking Brain Compression(SBC),将经典最优脑外科法扩展至SNN,采用可低成本计算海森矩阵的脉冲序列目标,仅需一次反向传播即可完成参数压缩与解析重标定。SBC在事件驱动和静态数据集(包括ImageNet)上应用于多种SNN结构(如SEW-ResNet152与脉冲驱动Transformer),实现当前最优的一次性后训练压缩效果,相较移植至SNN的ANN压缩基线,准确率提升达1至2位数。此外,我们报告了一种基于突触操作的能耗代理指标,并通过小样本校准消融实验,验证了其在每类少于一个样本条件下的稳健性能。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) have emerged as a new generation of energy-efficient neural networks suitable for implementation on neuromorphic hardware. As neuromorphic hardware has limited memory and computational resources, parameter pruning and quantization have recently been explored to improve the efficiency of SNNs. State-of-the-art SNN pruning/quantization methods employ multiple compression and training iterations, increasing the cost for pre-trained or very large SNNs. In this paper, we propose a novel one-shot post-training compression framework, Spiking Brain Compression (SBC), that extends the classical Optimal Brain Surgeon method to SNNs. SBC replaces the current-based objective found in the common layer-wise compression method with a spike-train-based objective whose Hessian is cheaply computable, allowing a single backward pass to compress parameters and analytically rescale the rest. Applying SBC to SNN pruning and quantization across event-based and static datasets (up to ImageNet), including SEW-ResNet152 and spike-driven Transformers, we achieve state-of-the-art one-shot post-training compression for SNNs, with single- to double-digit accuracy gains over ANN compression baselines ported to SNNs. We further report a synaptic-operation-based energy proxy and a calibration-size ablation, demonstrating robust performance under sub-one-sample-per-class calibration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。