用强化学习验证超级马里奥关卡设计的教育价值
Reinforcement Learning in Super Mario Bros: Curriculum, Pedagogy, and Optimal Level Design in World 1-1
- 构建离散环境测试四种算法在递进关卡中的表现
- 蒙特卡洛方法胜出，94.9%胜率且学会路径奖励最大化
- 原始关卡顺序最优，提升学习效率并避免失败
《超级马里奥兄弟》世界1-1被广泛视为游戏设计典范：其渐进结构被认为通过关卡本身传授核心机制。我们通过强化学习检验这一结构是否可量化。将世界1-1从零构建为完全离散环境，对比Q-Learning、SARSA、蒙特卡洛（Monte Carlo）和深度Q网络（DQN）四类算法在三个逐步复杂版本中的表现。蒙特卡洛以94.9% ± 1.5%的胜率领先，优于DQN的76.4% ± 3.4%，因其学会在获胜路径上最大化中间奖励，而非走最短路线。进一步使用蒙特卡洛在12种条件中对六个经典段落进行课程式排列实验，发现原始顺序收敛最快、学习效率最高，且唯一实现零灾难性失败；无随机排列能同时满足三项指标。结果表明，世界1-1的原始设计具有可测量的教育结构，显著加速学习过程，无法由随机生成复制。
原文摘要 · Abstract (English)
World 1-1 of Super Mario Bros is widely celebrated as a masterclass in game design: its progressive structure is credited with teaching players core mechanics through the level itself. We ask whether that structure is empirically measurable using reinforcement learning. We implement World 1-1 from scratch as a fully discrete environment and compare four algorithms -- Q-Learning, SARSA, Monte Carlo, and Deep Q-Network (DQN) -- across three progressively complex versions of the same level. Monte Carlo emerges as the strongest agent (94.9% $\pm$ 1.5% win rate), outperforming DQN (76.4% $\pm$ 3.4%) by learning to maximize intermediate rewards along winning paths rather than taking the most direct route. We then use Monte Carlo in a curriculum experiment permuting World 1-1's six canonical segments across twelve conditions. Canonical ordering converges fastest, achieves the highest learning efficiency, and is the only condition with zero catastrophic failures; no random permutation matches all three criteria simultaneously. These results provide, to the best of our knowledge, the first empirical validation that World 1-1's canonical design encodes genuine pedagogical structure: one that measurably accelerates learning and cannot be replicated by chance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。