对比卷积与注意力结构在强化学习中的表现，发现卷积模型更优。
Transforming Game Play: A Comparative Study of DCQN and DTQN Architectures in Reinforcement Learning
- 用卷积和注意力结构构建DQN，在Atari游戏上对比性能
- 3500万到4000万参数下，卷积模型速度更快、多数游戏表现更好
- 仅在Centipede游戏中注意力模型略胜，其余均落后
本研究在三个Atari游戏Asteroids、Space Invaders和Centipede上，比较了基于卷积神经网络（CNN）和Transformer架构的深度Q网络（DCQN与DTQN）的性能。深度Q网络（DQN）已显著推动强化学习发展，使智能体能直接从高维感官输入（如像素或RAM数据）中学习最优策略。尽管基于CNN的DQN已被广泛研究并应用于多个领域，但基于Transformer的DQN仍处于探索阶段。本研究旨在填补这一空白，通过基准测试比较两种架构的表现。结果表明，在3500万至4000万参数范围内，无论采用ViT还是投影结构，DCQN在速度和多数游戏表现上均优于DTQN。仅在Centipede游戏中，DTQN表现略好于DCQN。
原文摘要 · Abstract (English)
In this study, we investigate the performance of Deep Q-Networks utilizing Convolutional Neural Networks (CNNs) and Transformer architectures across three different Atari games. The advent of DQNs has significantly advanced Reinforcement Learning, enabling agents to directly learn optimal policies from high-dimensional sensory inputs from pixel or RAM data. While CNN-based DQNs have been extensively studied and deployed in various domains, Transformer-based DQNs are relatively unexplored. Our research aims to fill this gap by benchmarking the performance of both DCQNs and DTQNs across the Atari games Asteroids, Space Invaders, and Centipede. We find that in the 35-40 million parameter range, the DCQN outperforms the DTQN in speed across both ViT and Projection Architectures. We also find the DCQN outperforms the DTQN in all games except for Centipede.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。