针对脉冲神经网络加速，提出动态时间聚合方法，按数据类型选择压缩或保留时间信息。
Collapse or Preserve: Data-Dependent Temporal Aggregation for Spiking Neural Network Acceleration
- 先聚合K帧脉冲再做一次卷积，减少计算次数
- 在手写数字数据上提速13.8倍且准确率提升1.6%
- 对事件数据保留时间分辨率，避免信息丢失
脉冲稀疏性常被认为可提升脉冲神经网络在GPU上的推理效率。我们发现这只是一个假象：在Apple M3 Max上，五种不同的稀疏计算策略均未能超越稠密卷积，因为SIMD架构无法利用独立同分布二值脉冲的细粒度、无结构稀疏性。为此，我们提出时间聚合卷积（TAC），利用卷积线性特性，在单次卷积前预聚合K帧脉冲，将T次调用减少至T/K次。在速率编码数据上，TAC在MNIST上实现13.8倍加速且准确率提升1.6%，在Fashion-MNIST上提升5.4%，速度与精度同步改善。然而，在包含真实运动信息的事件数据中，时间聚合会导致信息损失。因此，我们引入TAC-TP（时间保持），将每组卷积输出共享给K个独立的LIF步，保留下游层的时间分辨率。在DVS128-Gesture数据集上，TAC-TP达到95.1%准确率（基线96.3%），卷积调用减少50%，而标准TAC降至91.3%。核心发现是：最优时间聚合策略依赖于数据类型——速率编码数据应压缩时间维度（降噪），事件数据应保留时间维度（保信息）。该加速机制与硬件无关：TAC在NVIDIA V100上实现11.0倍加速。mlx-snn库中所有算子开源。
原文摘要 · Abstract (English)
Spike sparsity is widely believed to enable efficient spiking neural network (SNN) inference on GPU hardware. We demonstrate this is an illusion: five distinct sparse computation strategies on Apple M3 Max all fail to outperform dense convolution, because SIMD architectures cannot exploit the fine-grained, unstructured sparsity of i.i.d. binary spikes. Instead, we propose Temporal Aggregated Convolution (TAC), which exploits convolution linearity to pre-aggregate $K$ spike frames before a single convolution call, reducing $T$ calls to $T/K$. On rate-coded data, TAC achieves 13.8times speedup with +1.6% accuracy on MNIST and +5.4% on Fashion-MNIST -- a simultaneous improvement in both speed and accuracy. However, on event-based data where the temporal dimension carries genuine motion information, TAC's temporal collapse is harmful. We therefore introduce TAC-TP (Temporal Preservation), which shares each group's convolution output across K independent LIF steps, preserving full temporal resolution for downstream layers. On DVS128-Gesture, TAC-TP achieves 95.1% accuracy (vs. 96.3% baseline) with 50% fewer convolution calls, while standard TAC drops to 91.3%. Our key finding is that the optimal temporal aggregation strategy is data-dependent: collapse the temporal dimension for rate-coded data (noise reduction) but preserve it for event data (information retention). Speedup is hardware-agnostic: TAC achieves 11.0times on NVIDIA V100, confirming the mechanism transfers across GPU architectures. All operators in the mlx-snn library are open source.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。