提出缓解神经网络持续学习中遗忘问题的新方法
Overcoming catastrophic forgetting in neural networks
- 采用弹性权重固化技术，保留旧知识同时学习新任务
- 在旋转变换和打乱的MNIST数据上显著降低遗忘率
- 适合需要长期学习的AI系统开发者参考
灾难性遗忘是持续学习中的核心挑战，指神经网络在顺序学习多个任务时无法保留已有知识。本文通过在扰动MNIST和旋转MNIST基准上复现并扩展弹性权重固化（EWC）方法，对比了其与L2正则化及无正则化随机梯度下降（SGD）的表现。结果表明，相比直接训练，EWC显著减少了遗忘，但略微降低了新任务的学习效率。同时研究了丢弃法正则化和不同超参数的影响，揭示了EWC在多种学习场景下的泛化能力。这些发现证实了EWC在实现神经网络终身学习中的可行性。
原文摘要 · Abstract (English)
Catastrophic forgetting is the primary challenge that hinders continual learning, which refers to a neural network ability to sequentially learn multiple tasks while retaining previously acquired knowledge. Elastic Weight Consolidation, a regularization-based approach inspired by synaptic consolidation in biological neural systems, has been used to overcome this problem. In this study prior research is replicated and extended by evaluating EWC in supervised learning settings using the PermutedMNIST and RotatedMNIST benchmarks. Through systematic comparisons with L2 regularization and stochastic gradient descent (SGD) without regularization, we analyze how different approaches balance knowledge retention and adaptability. Our results confirm what was shown in previous research, showing that EWC significantly reduces forgetting compared to naive training while slightly compromising learning efficiency on new tasks. Moreover, we investigate the impact of dropout regularization and varying hyperparameters, offering insights into the generalization of EWC across diverse learning scenarios. These results underscore EWC's potential as a viable solution for lifelong learning in neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。