arXiv:2505.10909cs.ARcs.AI2025-05被引 11

通过模式化分层稀疏性,显著提升脉冲神经网络的计算效率。

Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks

论文配图:Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networks
图 1 · 摘自论文原文
  • 利用预定义模式压缩脉冲激活,实现向量级稀疏化
  • 两级稀疏结构使推理速度提升3.45倍,能效提高4.93倍
  • 适合低功耗部署场景,尤其适用于边缘计算设备

脉冲神经网络(SNNs)因其能效优势和生物合理性受到关注,其计算依赖0-1脉冲激活带来的稀疏性。现有加速器虽利用该稀疏性跳过零值计算,却忽视了二进制激活中固有的分布模式。本文发现特定激活模式存在,并据此提出新型 extbf{基于模式的分层稀疏性}框架 extbf{ extit{Phi}}。该框架包含两层稀疏结构:第一层以预定义模式表示激活,支持离线权重预计算,大幅减少运行时开销;第二层在第一层基础上引入元素级稀疏矩阵,进一步降低计算量并保持精度。算法上采用k-means模式选择与模式感知微调技术;硬件上设计专用架构 extbf{ extit{Phi}},实时高效处理两级稀疏性。大量实验表明,相比最先进SNN加速器, extit{Phi}实现3.45倍加速与4.93倍能效提升,验证了其优化潜力。

原文摘要 · Abstract (English)

Spiking Neural Networks (SNNs) are gaining attention for their energy efficiency and biological plausibility, utilizing 0-1 activation sparsity through spike-driven computation. While existing SNN accelerators exploit this sparsity to skip zero computations, they often overlook the unique distribution patterns inherent in binary activations. In this work, we observe that particular patterns exist in spike activations, which we can utilize to reduce the substantial computation of SNN models. Based on these findings, we propose a novel \textbf{pattern-based hierarchical sparsity} framework, termed \textbf{\textit{Phi}}, to optimize computation. \textit{Phi} introduces a two-level sparsity hierarchy: Level 1 exhibits vector-wise sparsity by representing activations with pre-defined patterns, allowing for offline pre-computation with weights and significantly reducing most runtime computation. Level 2 features element-wise sparsity by complementing the Level 1 matrix, using a highly sparse matrix to further reduce computation while maintaining accuracy. We present an algorithm-hardware co-design approach. Algorithmically, we employ a k-means-based pattern selection method to identify representative patterns and introduce a pattern-aware fine-tuning technique to enhance Level 2 sparsity. Architecturally, we design \textbf{\textit{Phi}}, a dedicated hardware architecture that efficiently processes the two levels of \textit{Phi} sparsity on the fly. Extensive experiments demonstrate that \textit{Phi} achieves a $3.45\times$ speedup and a $4.93\times$ improvement in energy efficiency compared to state-of-the-art SNN accelerators, showcasing the effectiveness of our framework in optimizing SNN computation.

脉冲神经网络稀疏性优化能效提升硬件协同设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。