提出可学习参数的脉冲神经元,提升音频识别性能与梯度传播。
Gated Parametric Neuron for Spike-based Audio Recognition
- 引入门控机制的可调参数神经元,增强梯度流动。
- 在两个脉冲音频数据集上超越主流SNN模型,实现高精度识别。
- 适合研究脉冲神经网络与生物可塑性计算的学者。
脉冲神经网络(SNN)旨在模拟人脑中生物可解释的神经元。目前广泛研究的漏电积分-发放(LIF)神经元在反向传播训练时存在梯度消失问题,且其参数通常手动设定并固定,与真实神经元的异质性不符。本文提出一种门控可调参数神经元(GPN),通过门控机制有效处理时空信息。相较于LIF神经元,GPN具有两大优势:一是改善梯度传播,缓解梯度消失;二是能自动学习时空异质的神经元参数。此外,采用相同门结构消除初始参数设置,并设计混合循环神经网络-SNN结构。在两个脉冲音频数据集上的实验表明,GPN网络优于多个先进SNN模型,能有效缓解梯度消失,具备时空异质参数,展现出处理长时依赖并实现高性能的能力。
原文摘要 · Abstract (English)
Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures. However, it has the vanishing gradient problem when trained with backpropagation. Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: 1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, 2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。