用记忆回放和Transformer恢复模型持续学习中的适应能力
Experience Replay Addresses Loss of Plasticity in Continual Learning
- 引入经验回放作为记忆机制，结合Transformer处理旧数据
- 在回归、分类等任务中彻底消除性能退化现象
- 无需修改训练流程，适合研究持续学习的算法设计
深度神经网络在持续学习中面临的主要挑战之一是可塑性丧失：随着反向传播训练进行，模型逐渐失去适应新任务的能力，性能远低于随机初始化状态。本文提出新假设：经验回放（experience replay）通过提供记忆支持，缓解了这一问题。我们在多种任务（包括回归、分类和策略评估）中验证该假设，仅通过添加经验回放并用Transformer处理其数据，即可完全消除可塑性丧失。关键在于未改变任何标准深度学习组件——不修改反向传播、激活函数或使用正则化。我们推测，该机制有效源于Transformer的上下文学习能力。
原文摘要 · Abstract (English)
Loss of plasticity is one of the main challenges in continual learning with deep neural networks, where neural networks trained via backpropagation gradually lose their ability to adapt to new tasks and perform significantly worse than their freshly initialized counterparts. The main contribution of this paper is to propose a new hypothesis that experience replay addresses the loss of plasticity in continual learning. Here, experience replay is a form of memory. We provide supporting evidence for this hypothesis. In particular, we demonstrate in multiple different tasks, including regression, classification, and policy evaluation, that by simply adding an experience replay and processing the data in the experience replay with Transformers, the loss of plasticity disappears. Notably, we do not alter any standard components of deep learning. For example, we do not change backpropagation. We do not modify the activation functions. And we do not use any regularization. We conjecture that experience replay and Transformers can address the loss of plasticity because of the in-context learning phenomenon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。