通过简单架构改进，显著提升强化学习在ProcGen上的泛化能力。
Improving Generalization on the ProcGen Benchmark with Simple Architectural Changes and Scale
- 用3D卷积替代2D卷积，增加帧堆叠和卷积核数量。
- 优化后最优差距降低37.9%（从0.58降至0.36）。
- 方法简洁通用，适合追求高效泛化的研究者。
我们证明，结合近期强化学习进展与简单的架构改进，可显著提升在ProcGen基准上的泛化性能。具体改进包括帧堆叠、将2D卷积层替换为3D卷积层，以及每层增加卷积核数量。在所有环境中使用同一组超参数的实验结果表明，最优差距相比基线降低了37.9%（从0.58降至0.36），性能达到或超过当前最先进方法。这些改进具有高度正交性，可与现有泛化增强方法互补，提示该方向仍有较大提升空间。
原文摘要 · Abstract (English)
We demonstrate that recent advances in reinforcement learning (RL) combined with simple architectural changes significantly improves generalization on the ProcGen benchmark. These changes are frame stacking, replacing 2D convolutional layers with 3D convolutional layers, and scaling up the number of convolutional kernels per layer. Experimental results using a single set of hyperparameters across all environments show a 37.9\% reduction in the optimality gap compared to the baseline (from 0.58 to 0.36). This performance matches or exceeds current state-of-the-art methods. The proposed changes are largely orthogonal and therefore complementary to the existing approaches for improving generalization in RL, and our results suggest that further exploration in this direction could yield substantial improvements in addressing generalization challenges in deep reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。