arXiv:2510.04098cs.NEcs.AI2025-10被引 1

提出一种能高效训练脉冲神经网络的新数据剪枝方法。

Efficient Training of Spiking Neural Networks by Spike-aware Data Pruning

  • 根据脉冲响应设计重要性评分,降低梯度方差
  • 在不同剪枝率下实现接近理论极限的加速效果
  • 适合资源受限但需高效训练SNN的研究者

脉冲神经网络(SNN)作为传统人工神经网络(ANN)的节能替代方案,随着模型和数据集规模扩大而快速发展。然而,这种扩展带来了巨大的训练开销,对计算资源有限的研究者构成挑战,并阻碍了SNN的持续发展。数据剪枝通过保留最具信息量的样本、剔除冗余样本,是加速训练的有前景策略,但在SNN中仍处于探索阶段。直接将基于ANN的数据剪枝方法应用于SNN,无法捕捉样本内在重要性,且梯度方差高。为此,本文提出一种新型脉冲感知数据剪枝(SADP)方法。SADP通过将每个样本的选择概率与梯度范数成比例来减少梯度方差,同时利用一种高效上界——脉冲感知重要性得分,避免直接计算梯度带来的高昂成本。该得分考虑了全或无脉冲对梯度范数的影响,可近乎零开销计算。在多种数据集和架构上的大量实验表明,SADP始终优于现有剪枝基线,在不同剪枝率下均达到接近理论最大值的训练加速。值得注意的是,SADP在ImageNet上将训练时间缩短35%,同时保持与全数据训练相当的准确率。本工作建立了一种以数据为中心的高效SNN训练范式,为将SNN扩展至更大模型和数据集铺平道路。源代码将在评审后公开。

原文摘要 · Abstract (English)

Spiking neural networks (SNNs), recognized as an energy-efficient alternative to traditional artificial neural networks (ANNs), have advanced rapidly through the scaling of models and datasets. However, such scaling incurs considerable training overhead, posing challenges for researchers with limited computational resources and hindering the sustained development of SNNs. Data pruning is a promising strategy for accelerating training by retaining the most informative examples and discarding redundant ones, but it remains largely unexplored in SNNs. Directly applying ANN-based data pruning methods to SNNs fails to capture the intrinsic importance of examples and suffers from high gradient variance. To address these challenges, we propose a novel spike-aware data pruning (SADP) method. SADP reduces gradient variance by determining each example's selection probability to be proportional to its gradient norm, while avoiding the high cost of direct gradient computation through an efficient upper bound, termed spike-aware importance score. This score accounts for the influence of all-or-nothing spikes on the gradient norm and can be computed with negligible overhead. Extensive experiments across diverse datasets and architectures demonstrate that SADP consistently outperforms data pruning baselines and achieves training speedups close to the theoretical maxima at different pruning ratios. Notably, SADP reduces training time by 35% on ImageNet while maintaining accuracy comparable to that of full-data training. This work, therefore, establishes a data-centric paradigm for efficient SNN training and paves the way for scaling SNNs to larger models and datasets. The source code will be released publicly after the review process.

脉冲神经网络数据剪枝高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。