提出新剪枝方法，在保持精度前提下显著降低脉冲神经网络能耗。
Criticality-Constrained Iterative Pruning for Energy-Efficient Spiking Neural Networks via Combined Importance Scoring
- 融合权重大小与神经元临界性，构建精确重要性评分
- 90%稀疏度下准确率达95.6%，比传统方法高2.2个百分点
- 适合追求低功耗部署的脉冲神经网络研究者
在类脑硬件上部署脉冲神经网络（SNN）需激进的突触剪枝以节省能耗，同时保持时序计算完整性。现有方法或忽略神经元临界性，或依赖对固有组合优化问题的凸松弛，其分数掩码经二值化后会严重破坏精度。本文提出原生PyTorch的临界性约束二次剪枝（CQP）框架，将权重大小与代理梯度临界性融合为解析精确的重要性度量，消除求解器方法中普遍存在的舍入误差。我们首次揭示连续松弛陷阱：OSQP求解器的分数掩码可超出目标稀疏度达12个百分点，导致准确率暴跌44个百分点。识别并修复了Adam一阶矩张量复活已剪枝突触的“僵尸权重”失效模式，违反二值稀疏性保证。通过迭代流程——剪枝、带梯度掩码微调、重新计算临界性、重复——有效缓解高稀疏度下的梯度停滞。基于KL散度的时间分析发现冗余仿真步，无需修改权重即可理论节能10%。在MNIST（60,000个训练样本）上，CQP实现90%稀疏度下95.6%准确率，优于幅度剪枝的93.4%（+2.2个百分点）。临界性阈值扫描显示临界悬崖现象：当阈值达tau=0.9时，准确率从87.0%骤降至14.4%，为SNN层面的临界大脑假说提供量化支持。结合权重稀疏化与时间截断，实现70%稀疏度下每推理能耗降低73%，验证该流水线在类脑部署中的实用价值。
原文摘要 · Abstract (English)
Deploying spiking neural networks (SNNs) on neuromorphic hardware demands aggressive synaptic pruning while preserving temporal computation integrity. Existing strategies either neglect neuronal criticality or rely on convex relaxations of the inherently combinatorial pruning problem whose fractional masks, upon binarisation, destroy accuracy at moderate-to-high sparsity. We present Criticality-Constrained Quadratic Pruning (CQP), a native PyTorch pipeline that fuses weight magnitude with surrogate-gradient criticality into an analytically exact importance metric, eliminating the rounding artefacts endemic to solver-based approaches. We formally characterise a continuous-relaxation trap wherein OSQP-solver fractional masks overshoot the intended sparsity by up to 12 percentage points (pp), precipitating a 44 pp accuracy collapse. We identify and remediate a zombie-weight failure mode in which Adam's first-moment tensors resurrect pruned synapses, violating the binary sparsity guarantee. An iterative schedule - prune, fine-tune with gradient masking, recompute criticality, and repeat - eliminates gradient staleness at high sparsity. A KL-divergence temporal analysis identifies a redundant simulation timestep, enabling a free 10% theoretical energy reduction without weight modification. On MNIST (60,000 training examples), CQP yields 95.6% accuracy at 90% sparsity versus 93.4% for magnitude pruning (+2.2 pp). A criticality-threshold sweep reveals an empirical criticality cliff: accuracy falls from 87.0% to 14.4% as the threshold reaches tau = 0.9, constituting a quantitative SNN-level analogue of the Critical Brain Hypothesis. Combined weight sparsification and temporal truncation yield a compound 73% reduction in per-inference energy at 70% sparsity, confirming the practical value of the proposed pipeline for neuromorphic deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。