让循环神经网络在强化学习中高效运行，性能提升35.6%
Recurrent Off-Policy Deep Reinforcement Learning Doesn't Have to be Slow
- 用可学习与不可学习编码层结合，实现低开销的循环架构
- 在Atari上实现35.6%的人类归一化中位数性能提升
- 适用于主流非循环强化学习算法，适配性强
循环式离策略深度强化学习模型虽达顶尖性能，却常因计算开销大被弃用。为此，我们提出RISE（基于简化编码的循环整合），通过结合可学习与不可学习编码层，使循环网络可在任意图像输入的离策略强化学习场景中使用，且几乎无额外计算负担。将RISE集成至主流非循环离策略算法后，我们在Atari基准测试中观察到人类归一化四分位数均值（IQM）提升了35.6%。我们分析了多种实现策略，凸显该框架的通用性与潜力。
原文摘要 · Abstract (English)
Recurrent off-policy deep reinforcement learning models achieve state-of-the-art performance but are often sidelined due to their high computational demands. In response, we introduce RISE (Recurrent Integration via Simplified Encodings), a novel approach that can leverage recurrent networks in any image-based off-policy RL setting without significant computational overheads via using both learnable and non-learnable encoder layers. When integrating RISE into leading non-recurrent off-policy RL algorithms, we observe a 35.6% human-normalized interquartile mean (IQM) performance improvement across the Atari benchmark. We analyze various implementation strategies to highlight the versatility and potential of our proposed framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。