提出三种高效重启策略，显著降低非平稳强化学习的动态遗憾。
Efficient Restarts in Non-Stationary Model-Free Reinforcement Learning
- 部分、自适应、选择性重启，避免完全遗忘与固定时间重启
- 在多环境中动态遗憾降低最高达91%相比RestartQ-UCB
- 适合动态环境下的强化学习系统，尤其关注稳定性与适应性
本文针对无模型非平稳强化学习提出三种高效重启机制。我们指出Mao等（2022）提出的RestartQ-UCB算法存在两个核心问题：一是完全遗忘，重启后所有环境知识丢失；二是定时重启，无论策略是否适配当前环境动态，均按预设时间重启。为此，我们提出部分重启、自适应重启和选择性重启三种改进方法，应用于RestartQ-UCB与RANDOMIZEDQ（Wang等，2025）。实验显示，在多个不同环境中均实现近最优性能，动态遗憾相比RestartQ-UCB最高降低91%。
原文摘要 · Abstract (English)
In this work, we propose three efficient restart paradigms for model-free non-stationary reinforcement learning (RL). We identify two core issues with the restart design of Mao et al. (2022)'s RestartQ-UCB algorithm: (1) complete forgetting, where all the information learned about an environment is lost after a restart, and (2) scheduled restarts, in which restarts occur only at predefined timings, regardless of the incompatibility of the policy with the current environment dynamics. We introduce three approaches, which we call partial, adaptive, and selective restarts to modify the algorithms RestartQ-UCB and RANDOMIZEDQ (Wang et al., 2025). We find near-optimal empirical performance in multiple different environments, decreasing dynamic regret by up to $91$% relative to RestartQ-UCB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。