用强化学习选学习率，动态适配训练过程。
Dynamic Learning Rate for Deep Reinforcement Learning: A Bandit Approach
- 基于策略表现自适应选择学习率，不依赖固定步数
- 在Atari和MuJoCo上表现优于或媲美调优基线
- 适合应对非平稳目标的深度强化学习任务
在深度强化学习中，学习率对稳定性和性能至关重要，但其最优值会随环境和策略演化而变化。标准衰减调度器假设单调收敛，常与实际动态脱节，导致调整过早或过晚。我们提出LRRL，一种元学习方法，根据策略表现动态选择学习率，而非依赖训练步数。LRRL自适应地偏好能提升回报的学习率，在候选集包含个别导致发散的值时仍保持鲁棒。在Atari和MuJoCo基准测试中，LRRL性能达到或超过调优基线和标准调度器。结果表明，LRRL是应对深度强化学习中非平稳目标的实用解决方案。
原文摘要 · Abstract (English)
In deep Reinforcement Learning (RL), the learning rate critically influences both stability and performance, yet its optimal value shifts during training as the environment and policy evolve. Standard decay schedulers assume monotonic convergence and often misalign with these dynamics, leading to premature or delayed adjustments. We introduce LRRL, a meta-learning approach that dynamically selects the learning rate based on policy performance rather than training steps. LRRL adaptively favors rates that improve returns, remaining robust even when the candidate set includes values that individually cause divergence. Across Atari and MuJoCo benchmarks, LRRL achieves performance competitive with or superior to tuned baselines and standard schedulers. Our findings position LRRL as a practical solution for adapting to non-stationary objectives in deep RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。