对比DT与DM在Atari游戏中的表现,发现复杂度决定优劣。
Decision Transformer vs. Decision Mamba: Analysing the Complexity of Sequential Decision Making in Atari Games
- 通过分析12个游戏的特性,量化动作与视觉复杂度对模型表现的影响。
- 动作空间和视觉复杂度越高,Decision Transformer表现越优,反之Mamba更胜一筹。
- 研究结果可指导复杂环境下的模型选型与设计。
本研究分析了Decision Transformer(DT)与Decision Mamba(DM)在不同Atari游戏中序列建模强化学习任务的表现差异。实验扩展至12个游戏,发现DM在Breakout和Qbert等简单游戏中表现更佳,而DT在Hero和Kung Fu Master等复杂游戏中领先。通过评估动作空间复杂度、视觉复杂度、平均轨迹长度及首次非零奖励步数等特征,结合压缩比衡量视觉复杂度、随机森林回归与相关性分析,结果表明性能差距由多重因素交互决定,其中动作空间复杂度和视觉复杂度(以压缩比为指标)是主要影响因素。在动作与视觉元素简单的环境中DM占优,而在高复杂度环境中DT更具优势。研究深化了对游戏特性如何影响序列建模强化学习性能的理解,为未来模型设计与应用提供参考。
原文摘要 · Abstract (English)
This work analyses the disparity in performance between Decision Transformer (DT) and Decision Mamba (DM) in sequence modelling reinforcement learning tasks for different Atari games. The study first observed that DM generally outperformed DT in the games Breakout and Qbert, while DT performed better in more complicated games, such as Hero and Kung Fu Master. To understand these differences, we expanded the number of games to 12 and performed a comprehensive analysis of game characteristics, including action space complexity, visual complexity, average trajectory length, and average steps to the first non-zero reward. In order to further analyse the key factors that impact the disparity in performance between DT and DM, we employ various approaches, including quantifying visual complexity, random forest regression, correlation analysis, and action space simplification strategies. The results indicate that the performance gap between DT and DM is affected by the complex interaction of multiple factors, with the complexity of the action space and visual complexity (particularly evaluated by compression ratio) being the primary determining factors. DM performs well in environments with simple action and visual elements, while DT shows an advantage in games with higher action and visual complexity. Our findings contribute to a deeper understanding of how the game characteristics affect the performance difference in sequential modelling reinforcement learning, potentially guiding the development of future model design and applications for diverse and complex environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。