用大模型自动生成可训练的网页游戏，让强化学习环境构建快如闪电。
PlayTrain: An Efficient Reinforcement Learning Framework for LLM-Generated Adaptable JavaScript Games
- 大模型生成JavaScript游戏代码，一键转换为标准RL环境
- 单卡每秒训练超100万次决策，速度远超传统方法
- 适合游戏开发、RL研究者快速搭建定制化测试场景
尽管视频游戏环境在推动强化学习发展方面作用关键，但开发新环境或修改现有环境仍需大量手动编码。本文提出PlayTrain，一个结合大语言模型生成高质量JavaScript游戏代码，并通过高效流水线将任意JS游戏转化为标准gym环境的强化学习框架。利用大模型生成代码的能力和JS格式的可执行性，用户仅需少量提示即可生成可运行的游戏，且可直接用于训练代理。我们展示了多个应用场景：用简洁的JS克隆经典Atari与ProcGen游戏，在单个GPU节点上实现像素级代理端到端训练，速度超过100万次/秒；同时支持创建带新测试集、程序化生成逻辑或动态机制的变体。通过PlayTrain，只需一个由大模型生成并修改的JS文件，即可完成整个游戏环境开发。该框架为未来强化学习研究提供了新方向。
原文摘要 · Abstract (English)
While many video-game environments (VGEs) have played crucial roles in advancing reinforcement learning (RL), developing novel VGEs or modifying existing ones to support new features, has been a laborious process requiring extensive hand-coding. Here we present PlayTrain, an RL framework that combines the abilities of large language models (LLMs) to robustly generate JavaScript (JS) games from a minimal human prompt, and an efficient pipeline that can run any JS game in a standard 'gym' environment. Not only are recent LLMs particularly good at writing JS code, but the JS format also allows users to easily play generated VGEs, while PlayTrain enables us to train RL agents on the exact same games. We demonstrate multiple use cases of PlayTrain, including cloning well-known Atari and ProcGen games in simple JS, where PlayTrain trains pixel-based agents end-to-end at over 1M agent-decisions per second on a single GPU node; and creating modified versions thereof (e.g., that support novel test sets, procedural generation logics, or game dynamics). Through PlayTrain, we reimagine RL VGE development: all we need is a single JS file, generated and modified through an LLM. We discuss promising future RL research directions that PlayTrain unlocks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。