提出动态扩展脉冲神经网络,解决增量学习中旧知识遗忘问题。
ALADE-SNN: Adaptive Logit Alignment in Dynamically Expandable Spiking Neural Networks for Class Incremental Learning
- 通过自适应对齐输出逻辑值,平衡新旧任务特征表示。
- 在CIFAR100-B0上实现75.42%平均增量准确率,优于现有脉冲神经网络方法。
- 适合需要低功耗、持续学习的类脑计算场景。
受人类大脑在不遗忘旧知识的前提下学习新任务能力的启发,我们构建了具有动态结构的脉冲神经网络(SNN)以应对类别增量学习(CIL)挑战。对比实验表明,有限数据会导致不同任务间输出逻辑值分布偏差;固定旧任务特征提取器会引发过拟合,阻碍新任务学习。为此,我们提出ALADE-SNN框架,包含自适应对齐输出逻辑值以实现特征表示均衡,以及OtoN抑制机制,在训练时管理旧特征到新类别的权重映射,微调阶段释放这些权重。该方法基于分析结果动态调整网络结构,提升特征提取能力,并平衡新旧任务表现。实验显示,ALADE-SNN在CIFAR100-B0基准上经过10次增量步骤后,平均增量准确率达75.42%。该方法不仅达到基于深度神经网络(DNN)方法的性能水平,还超越现有最先进的基于脉冲神经网络的持续学习算法,推动了类脑计算中的持续学习发展,为实时数据处理提供了一种类脑、低功耗解决方案。
原文摘要 · Abstract (English)
Inspired by the human brain's ability to adapt to new tasks without erasing prior knowledge, we develop spiking neural networks (SNNs) with dynamic structures for Class Incremental Learning (CIL). Our comparative experiments reveal that limited datasets introduce biases in logits distributions among tasks. Fixed features from frozen past-task extractors can cause overfitting and hinder the learning of new tasks. To address these challenges, we propose the ALADE-SNN framework, which includes adaptive logit alignment for balanced feature representation and OtoN suppression to manage weights mapping frozen old features to new classes during training, releasing them during fine-tuning. This approach dynamically adjusts the network architecture based on analytical observations, improving feature extraction and balancing performance between new and old tasks. Experiment results show that ALADE-SNN achieves an average incremental accuracy of 75.42 on the CIFAR100-B0 benchmark over 10 incremental steps. ALADE-SNN not only matches the performance of DNN-based methods but also surpasses state-of-the-art SNN-based continual learning algorithms. This advancement enhances continual learning in neuromorphic computing, offering a brain-inspired, energy-efficient solution for real-time data processing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。