arXiv:2411.05802cs.NEcs.AI2024-11被引 6

根据任务相似性动态重用或扩展神经元,提升脉冲神经网络持续学习效率。

Similarity-based context aware continual learning for spiking neural networks

  • 按任务相似性智能重用旧神经元,相似度越高重用越多
  • 新任务所需新增神经元随相似度升高而减少,降低能耗
  • 适合追求生物可解释性与能效的持续学习研究者

生物大脑能根据任务上下文自适应地协调相关神经元群体,在真实环境中持续学习动态变化的任务。然而,现有基于脉冲神经网络的持续学习算法对各任务一视同仁,忽视了任务间相似性关联对网络学习的引导作用,限制了知识利用效率。受大脑上下文依赖可塑性机制启发,我们提出一种基于相似性的上下文感知脉冲神经网络(SCA-SNN)持续学习算法,以高效实现任务增量学习与类别增量学习。该模型基于任务间的上下文相似性,自适应地重用对新任务有益的旧神经元(相似度越高,重用越多),并灵活扩展新神经元(相似度越高,扩展越少)。选择性重用与区分性扩展显著提升了旧知识利用率并降低能耗。在CIFAR100、ImageNet广义数据集,以及FMNIST-MNIST、SVHN-CIFAR100混合数据集上的大量实验表明,SCA-SNN在性能上优于多种SNN与DNN基持续学习算法。此外,该算法可自适应为相关任务选择相似神经元组,为提升高效持续学习的生物可解释性提供了可行路径。

原文摘要 · Abstract (English)

Biological brains have the capability to adaptively coordinate relevant neuronal populations based on the task context to learn continuously changing tasks in real-world environments. However, existing spiking neural network-based continual learning algorithms treat each task equally, ignoring the guiding role of different task similarity associations for network learning, which limits knowledge utilization efficiency. Inspired by the context-dependent plasticity mechanism of the brain, we propose a Similarity-based Context Aware Spiking Neural Network (SCA-SNN) continual learning algorithm to efficiently accomplish task incremental learning and class incremental learning. Based on contextual similarity across tasks, the SCA-SNN model can adaptively reuse neurons from previous tasks that are beneficial for new tasks (the more similar, the more neurons are reused) and flexibly expand new neurons for the new task (the more similar, the fewer neurons are expanded). Selective reuse and discriminative expansion significantly improve the utilization of previous knowledge and reduce energy consumption. Extensive experimental results on CIFAR100, ImageNet generalized datasets, and FMNIST-MNIST, SVHN-CIFAR100 mixed datasets show that our SCA-SNN model achieves superior performance compared to both SNN-based and DNN-based continual learning algorithms. Additionally, our algorithm has the capability to adaptively select similar groups of neurons for related tasks, offering a promising approach to enhancing the biological interpretability of efficient continual learning.

脉冲神经网络持续学习生物可解释性能效优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。