用模拟睡眠的无监督重放提升小样本持续学习性能
Unsupervised Replay Strategies for Continual Learning with Limited Data
- 引入随机激活与局部赫布学习的睡眠阶段进行无监督重放
- 在少量数据下训练时准确率显著提升，有效缓解灾难性遗忘
- 适合研究持续学习与生物启发神经网络的学者
人工神经网络在数据稀缺或不均衡时表现有限，且在持续学习中面临新任务训练后遗忘旧知识的问题。人类大脑则能从少量例子中持续学习。本研究探索了‘睡眠’——一种包含随机激活与局部赫布学习规则的无监督阶段——对增量训练下有限且不平衡数据集（如MNIST和Fashion MNIST）的影响。结果发现，引入睡眠阶段显著提升了模型在有限数据下的准确率。当少数任务顺序训练时，睡眠重放不仅挽救了因新任务训练而被灾难性遗忘的旧知识，还常增强旧任务的性能，尤其在数据受限情况下效果更明显。该研究揭示了睡眠重放在提升学习效率与促进神经网络持续学习中的多重作用。
原文摘要 · Abstract (English)
Artificial neural networks (ANNs) show limited performance with scarce or imbalanced training data and face challenges with continuous learning, such as forgetting previously learned data after new tasks training. In contrast, the human brain can learn continuously and from just a few examples. This research explores the impact of 'sleep', an unsupervised phase incorporating stochastic activation with local Hebbian learning rules, on ANNs trained incrementally with limited and imbalanced datasets, specifically MNIST and Fashion MNIST. We discovered that introducing a sleep phase significantly enhanced accuracy in models trained with limited data. When a few tasks were trained sequentially, sleep replay not only rescued previously learned information that had been catastrophically forgetting following new task training but often enhanced performance in prior tasks, especially those trained with limited data. This study highlights the multifaceted role of sleep replay in augmenting learning efficiency and facilitating continual learning in ANNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。