ATLAS通过抽象后继特征实现快速适应，解决持续学习中的遗忘与低效问题。
ATLAS: Adaptive Topological Learning with Abstract Successors for Continual Learning
- 用动态生长网络与后继特征解耦状态转移与奖励信号
- 在非平稳环境中实现近瞬时目标切换，且有正向迁移效果
- 适合需要快速适应新任务的强化学习场景
当前无模型强化学习算法虽性能优异，但样本效率低且对环境变化不鲁棒；而有模型方法虽样本效率高，仍无法应对环境突变。本文提出自适应拓扑学习与抽象后继特征（ATLAS），结合需时生长网络与后继特征，在保持高样本效率的同时有效缓解灾难性遗忘。在空间导航任务中，对比主流在线与离线策略算法，实验表明通过结构上解耦转移动态与奖励信号，ATLAS可实现近瞬时的新目标适应，并展现出正向后向迁移，显著优于基线方法在非平稳环境中的表现。
原文摘要 · Abstract (English)
Contemporary model-free reinforcement learning algorithms can achieve very high performance, but have low sample efficiency and are not robust to changes in the environment. Model-based algorithms have much higher sample efficiency, but still fail when the environment shifts. This paper introduces Adaptive Topological Learning with Abstract Successors (ATLAS) to combat these challenges. ATLAS uses a Grow When Required network with Successor Features in order to achieve high sample efficiency while also robustly tackling catastrophic forgetting. We evaluate ATLAS in spatial navigation tasks, benchmarking its performance against common on-policy and off-policy algorithms. Our empirical results demonstrate that by structurally decoupling transition dynamics from the reward signal, ATLAS achieves near-instantaneous adaptation to new goals and can exhibit positive backward transfer, significantly outperforming baseline methods in non-stationary environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。