六项强化学习改进整合，桌面电脑12小时完成高分游戏训练
Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC
- 集成六项前沿RL优化，构建高效新算法
- 在Atari-60上达7.4的人类相对得分（IQM）
- 适配3D游戏，仅微调即可玩超级马里奥等复杂游戏
Rainbow DQN通过结合多项独立改进显著提升了强化学习智能体性能。本文提出「Beyond The Rainbow」（BTR），将来自强化学习文献的六项改进整合至Rainbow DQN，实现了基于桌面电脑的最新性能纪录，在Atari-60上达到7.4的人类相对四分位均值（IQM）。超越Atari，BTR成功用于复杂3D游戏训练，仅需少量算法调整即可在超级马里奥银河、马力欧卡丁车和真人快打中完成学习。设计时注重计算效率，使用高端桌面电脑在12小时内完成2亿个Atari帧的训练。此外，我们对各组件进行了详细消融实验，通过多种指标分析其性能与影响。代码已开源：https://github.com/VIPTankz/BTR。
原文摘要 · Abstract (English)
Rainbow Deep Q-Network (DQN) demonstrated combining multiple independent enhancements could significantly boost a reinforcement learning (RL) agent's performance. In this paper, we present "Beyond The Rainbow" (BTR), a novel algorithm that integrates six improvements from across the RL literature to Rainbow DQN, establishing a new state-of-the-art for RL using a desktop PC, with a human-normalized interquartile mean (IQM) of 7.4 on Atari-60. Beyond Atari, we demonstrate BTR's capability to handle complex 3D games, successfully training agents to play Super Mario Galaxy, Mario Kart, and Mortal Kombat with minimal algorithmic changes. Designing BTR with computational efficiency in mind, agents can be trained using a high-end desktop PC on 200 million Atari frames within 12 hours. Additionally, we conduct detailed ablation studies of each component, analyzing the performance and impact using numerous measures. Code is available at https://github.com/VIPTankz/BTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。