自动构建高性能强化学习环境，无需手工编码。
Automatic Generation of High-Performance RL Environments
- 用提示模板+多层验证+迭代修复生成环境
- 5个环境性能达基准水平，开销低于训练时间4%
- 可复现私有环境，解决数据污染担忧
将复杂的强化学习（RL）环境转化为高性能实现，传统上需数月专业工程。我们提出一种闭环方法，以极低计算成本生成等效的高性能环境。该方法采用通用提示模板、分层验证（属性、交互、轨迹测试）、迭代修复以及跨后端策略迁移，确保无仿真到仿真差距。我们在五个环境中演示三种工作流：（1）直接转换（无先前高性能实现），从 Game Boy 模拟器 PyBoy 转换至 EmuRust（通过 Rust IPC），以及从 Pokemon Showdown 转换至 PokeJAX（通过 JAX）；（2）与现有高性能实现对比验证，通过吞吐量与 Puffer Pong、MJX、Brax 在相同 GPU 批量大小下匹配；（3）新环境创建：基于网络提取规范生成首个 Pokemon TCG Pocket 环境 TCGJax。在 200M 参数规模下，环境开销低于训练时间的 4%。闭环方法确认所有五个环境等价。TCGJax 由未公开的私有参考生成，作为代理预训练数据污染的对照组。
原文摘要 · Abstract (English)
Translating complex reinforcement learning (RL) environments into high-performance implementations has traditionally required months of specialized engineering. We present a closed-loop methodology that produces equivalent high-performance environments for minimal compute cost. Our method uses a generic prompt template, hierarchical verification (property, interaction, and rollout tests), iterative repair, and cross-backend policy transfer to verify no sim-to-sim gap. We demonstrate three distinct workflows across five environments: (1) Direct translation (no prior performance implementation exists) from Game Boy emulator PyBoy to our EmuRust (via Rust IPC) and from Pokemon Showdown to our PokeJAX (via JAX); (2) Translation verified against existing performance implementations via throughput parity with Puffer Pong, MJX and Brax at matched GPU batch sizes; and (3) New environment creation: TCGJax, the first Pokemon TCG Pocket environment, created from a web-extracted specification. At 200M parameters, the environment overhead drops below 4% of training time. Our closed-loop methodology confirms equivalence for all five environments. TCGJax, synthesized from a private reference absent from public repositories, serves as a contamination control for agent pretraining data concerns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。