揭示离散时间脉冲网络的表达能力，量化其逼近连续函数所需的规模。
Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time
- 基于漏电整合-放电模型，分析离散时间脉冲网络的分段常数表示特性。
- 证明网络需至少3层、12个神经元才能逼近任意连续函数。
- 强调时间延迟对输入空间划分复杂度的关键作用，适合神经形态计算研究者。
近年来，脉冲神经网络（SNNs）作为应对传统人工神经网络（ANNs）能耗问题的潜在解决方案取得显著进展，但其理论理解仍远落后于相关文献数量。本文研究基于漏电整合-放电（LIF）神经元的离散时间模型，即离散时间LIF-SNNs，该框架虽广泛应用，却缺乏坚实的理论基础。我们证明：具有静态输入输出的离散时间LIF-SNNs可实现定义在多面体区域上的分段常数函数；更重要的是，我们量化了逼近连续函数所需的网络规模。此外，我们分析了延迟（时间步数）和深度（层数）对输入空间划分复杂度的影响。结果表明，延迟在决定网络表达力方面起关键作用，且与采用分段线性激活函数的ANN形成鲜明对比。最后，通过数值实验验证了理论结论。
原文摘要 · Abstract (English)
Recent years have seen significant progress in developing spiking neural networks (SNNs) as a potential solution to the energy challenges posed by conventional artificial neural networks (ANNs). However, our theoretical understanding of SNNs remains relatively limited compared to the ever-growing body of literature on ANNs. In this paper, we study a discrete-time model of SNNs based on leaky integrate-and-fire (LIF) neurons, referred to as discrete-time LIF-SNNs, a widely used framework that still lacks solid theoretical foundations. We demonstrate that discrete-time LIF-SNNs with static inputs and outputs realize piecewise constant functions defined on polyhedral regions, and more importantly, we quantify the network size required to approximate continuous functions. Moreover, we investigate the impact of latency (number of time steps) and depth (number of layers) on the complexity of the input space partitioning induced by discrete-time LIF-SNNs. Our analysis highlights the importance of latency and contrasts these networks with ANNs employing piecewise linear activation functions. Finally, we present numerical experiments to support our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。