用能量模型判断何时干预，让旧策略只在熟悉状态中指导新任务。
Energy-Based Transfer for Reinforcement Learning
- 基于能量分数识别分布外状态，仅在教师熟悉的区域提供指导。
- 在多任务和单任务设置中，样本效率提升显著，奖励更高。
- 适合需要持续学习的复杂环境，避免错误引导探索。
强化学习算法常因样本效率低而难以应用于多任务或持续学习场景。通过将先前训练好的教师策略知识迁移，可引导新相关任务的探索。但若新任务与教师训练任务差异过大，迁移指导可能次优，导致探索偏向低回报行为。本文提出一种基于能量的迁移学习方法，利用分布外检测机制选择性地发出指导信号，使教师仅在自身训练分布内的状态中介入。理论上证明能量得分反映教师的状态访问密度，实证显示在单任务与多任务设置下均提升了样本效率和性能。
原文摘要 · Abstract (English)
Reinforcement learning algorithms often suffer from poor sample efficiency, making them challenging to apply in multi-task or continual learning settings. Efficiency can be improved by transferring knowledge from a previously trained teacher policy to guide exploration in new but related tasks. However, if the new task sufficiently differs from the teacher's training task, the transferred guidance may be sub-optimal and bias exploration toward low-reward behaviors. We propose an energy-based transfer learning method that uses out-of-distribution detection to selectively issue guidance, enabling the teacher to intervene only in states within its training distribution. We theoretically show that energy scores reflect the teacher's state-visitation density and empirically demonstrate improved sample efficiency and performance across both single-task and multi-task settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。