提出双学习者框架，实现持续强化学习中知识高效积累与迁移。
Principled Fast and Meta Knowledge Learners for Continual Reinforcement Learning
- 快学习者负责快速迁移知识，元学习者防止遗忘并逐步整合新经验。
- 在多个像素和连续控制任务上优于基线方法，显著减少灾难性遗忘。
- 适合需要长期适应新环境的强化学习应用，如机器人控制与智能体训练。
受人类学习与记忆系统（特别是海马体与大脑皮层的相互作用）启发，本文提出一种由快学习者与元学习者组成的双学习者框架，以应对持续强化学习问题。两者协同工作，分别承担知识迁移与知识整合的任务：快学习者专注于快速知识转移，而元学习者通过显式最小化灾难性遗忘，逐步整合新经验，从而支持快学习者高效累积知识。为加速新环境下的适应，引入自适应元预热机制，选择性利用过往知识。在多种基于像素及连续控制的基准测试中，该双学习者方法表现优于基线方法。代码已开源：https://github.com/datake/FAME。
原文摘要 · Abstract (English)
Inspired by the human learning and memory system, particularly the interplay between the hippocampus and cerebral cortex, this study proposes a dual-learner framework comprising a fast learner and a meta learner to address continual Reinforcement Learning~(RL) problems. These two learners are coupled to perform distinct yet complementary roles: the fast learner focuses on knowledge transfer, while the meta learner ensures knowledge integration. In contrast to traditional multi-task RL approaches that share knowledge through average return maximization, our meta learner incrementally integrates new experiences by explicitly minimizing catastrophic forgetting, thereby supporting efficient cumulative knowledge transfer for the fast learner. To facilitate rapid adaptation in new environments, we introduce an adaptive meta warm-up mechanism that selectively harnesses past knowledge. We conduct experiments in various pixel-based and continuous control benchmarks, revealing the superior performance of continual learning for our proposed dual-learner approach relative to baseline methods. The code is released in https://github.com/datake/FAME.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。