在Doom游戏中对比Transformer与传统方法，发现后者表现更优。
Do We Need Transformers to Play FPS Video Games?
- 用Transformer改进Q-learning和离线策略学习
- 在VizDoom中传统方法优于Transformer模型
- 适合关注强化学习模型效率的研究者
本文研究了基于Transformer的架构在部分可观测环境VizDoom中的在线与离线强化学习表现。采用两种方法：用于在线学习的Deep Transformer Q-Network（DTQN）和用于离线学习的Decision Transformer（DT）。DTQN利用Transformer的序列建模能力提升部分可观测环境下的Q-learning性能；DT则将序列建模技术用于从历史轨迹中学习，无需与环境交互。实验表明，尽管Transformer在Atari游戏中表现良好，但在VizDoom环境中，传统方法仍优于基于Transformer的模型。
原文摘要 · Abstract (English)
In this paper, we explore the Transformer based architectures for reinforcement learning in both online and offline settings within the Doom game environment. Our investigation focuses on two primary approaches: Deep Transformer Q- learning Networks (DTQN) for online learning and Decision Transformers (DT) for offline reinforcement learning. DTQN leverages the sequential modelling capabilities of Transformers to enhance Q-learning in partially observable environments,while Decision Transformers repurpose sequence modelling techniques to enable offline agents to learn from past trajectories without direct interaction with the environment. We conclude that while Transformers might have performed well in Atari games, more traditional methods perform better than Transformer based method in both the settings in the VizDoom environment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。