让强化学习智能体在奖励和动作变化时自动适应，无需重训。
Adapting the Behavior of Reinforcement Learning Agents to Changing Action Spaces and Reward Functions
- 通过检测概念漂移动态调整学习参数，实现自适应更新。
- 在网格世界和交通信号仿真中，学习效率提升最高1.7倍。
- 适合需要持续学习的动态环境应用，如自动驾驶、机器人控制。
强化学习智能体在现实场景中常面临环境非平稳的问题，尤其当奖励函数变化或可用动作空间扩展时表现不佳。本文提出MORPHIN框架，一种自适应的Q-learning方法，可在不重新训练的情况下实现即时适应。该框架结合概念漂移检测与学习率、探索参数的动态调整，在保留已有策略知识的同时，应对奖励函数变化及动作空间的实时扩展。我们在网格世界基准和交通信号控制模拟中验证了该方法。结果表明，相比标准Q-learning基线，MORPHIN具有更快的收敛速度和持续适应能力，学习效率最高提升1.7倍。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) agents often struggle in real-world applications where environmental conditions are non-stationary, particularly when reward functions shift or the available action space expands. This paper introduces MORPHIN, a self-adaptive Q-learning framework that enables on-the-fly adaptation without full retraining. By integrating concept drift detection with dynamic adjustments to learning and exploration hyperparameters, MORPHIN adapts agents to changes in both the reward function and on-the-fly expansions of the agent's action space, while preserving prior policy knowledge to prevent catastrophic forgetting. We validate our approach using a Gridworld benchmark and a traffic signal control simulation. The results demonstrate that MORPHIN achieves superior convergence speed and continuous adaptation compared to a standard Q-learning baseline, improving learning efficiency by up to 1.7x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。