提出双层内在可塑性机制,让脉冲神经网络实现持续加速学习。
IP$^{2}$-RSNN: Bi-level Intrinsic Plasticity Enables Learning-to-learn in Recurrent Spiking Neural Networks
- 设计双层内在可塑性:慢速元调节可学习属性,快速调整任务内参数。
- 在多任务学习中,该模型比点神经元网络和自注意力模型更快适应新任务。
- 揭示了神经元与网络层面的任务特异性适应机制,适用于类脑深度学习研究。
学习到学习(L2L)是指在相似任务间逐步加快学习速度,是神经科学与人工智能的核心问题。然而,其神经机制仍不明确,因多数研究关注突触可塑性引发的神经群体动态,忽视了内在神经可塑性的作用,而点神经元模型无法捕捉此现象。为此,我们构建了具有双层内在可塑性(IP²-RSNN)的循环脉冲神经网络。首先,基于任务需求,慢速元内在可塑性决定哪些神经属性可被学习,并在后续任务中保持配置;其次,快速内在可塑性在每个任务内微调这些可学习属性。结果表明,该机制在循环脉冲神经网络中对实现L2L至关重要,且IP²-RSNN优于点神经元循环神经网络与自注意力模型。多尺度神经动态分析显示,双层内在可塑性对任务类型特异性适应至关重要,而点神经元模型无法捕捉此类适应。结果表明,内在可塑性为学习到学习提供了显著计算优势,为类脑深度学习模型的设计提供启示。
原文摘要 · Abstract (English)
Learning-to-learn (L2L), defined as progressively faster learning across similar tasks, is fundamental to both neuroscience and artificial intelligence. However, its neural basis remains elusive, as most studies emphasize neural population dynamics induced by synaptic plasticity while overlooking adaptations driven by intrinsic neuronal plasticity, which point-neuron models cannot capture. To address the above issue, we develop a recurrent spiking neural network with bi-level intrinsic plasticity (IP$^{2}$-RSNN). First, based on task demands, a slow meta-intrinsic plasticity determines which intrinsic neuronal properties are learnable, which is preserved throughout subsequent task learning once configured. Second, a fast intrinsic plasticity fine-tunes those learnable properties within each task. Our results indicate that the proposed bi-level intrinsic plasticity plays a critical role in enabling L2L in RSNNs and show that IP$^{2}$-RSNNs outperform point-neuron recurrent neural networks and self-attention models. Furthermore, our analysis of multi-scale neural dynamics reveals that the bi-level intrinsic plasticity is essential to task-type-specific adaptations at both the neuronal and network levels during L2L, while such adaptations cannot be captured by point-neuron models. Our results suggest that intrinsic plasticity provides significant computational advantages in L2L, shedding light on the design of brain-inspired deep learning models and algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。