TerraZero用程序化生成模拟器实现无示范自对弈训练,速度超快且泛化能力强。
TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale

- 通过程序化生成地图与交通参与者,实现零示范自对弈训练
- 单卡每秒处理130万智能体步数,支持大规模强化学习
- 无需人类示范或轨迹,可零样本跨城市泛化,适用于车/卡车/行人
训练鲁棒的自动驾驶智能体需要一个足够快、足够真实、足够多样化的模拟器。我们提出TerraZero,一个程序化驾驶模拟器与自对弈训练框架。其配置化的C语言引擎在CPU上运行仿真,同时在GPU上进行策略推理,通过零拷贝路径实现单服务器级GPU每秒130万代理步数,远超现有物体级模拟器。它保留了异构代理、多动态模型和完整交通规则执行等真实要素,而轻量级系统常忽略这些。仅以真实地图几何作为输入,地图内随机生成基于规则的路权使用者与信号控制器,并在每轮中随机化代理动态、奖励与尺寸,使单一地图产生近乎无限的场景。所有报告策略均从零开始纯强化学习训练,无任何人类示范、模仿、历史轨迹或推理时的备用规划器。该方法计算高效,可在多GPU上扩展。策略在未见过的城市和数据集上实现零样本泛化,包括未显式监督的左侧行驶。作为主智体,单个检查点是目前已知首个在val14和交互长尾评测InterPlan上均排名第一的完全学习型策略。在Waymo Open Sim Agents上,相同方法优于其他无示范方法,媲美最强参考锚定自对弈方法。一套系统同时提供最先进的无示范驾驶策略及协同控制车辆、行人与骑行者的模拟智能体。
原文摘要 · Abstract (English)
Training robust autonomous driving agents requires a simulator fast enough for reinforcement learning at scale, realistic enough to ground behavior in real-world map structure, and diverse enough to cover the safety-critical long tail that logged data rarely contains. We present TerraZero, a procedural driving simulator and self-play training stack that meets these goals. A configurable C engine runs simulation on the CPU and policy inference on the GPU over a zero-copy path, sustaining 1.3M agent-steps per second on a single server-grade GPU, far faster than existing object-level simulators, while keeping fidelity lighter single-agent systems omit: heterogeneous agents, multiple dynamics models, and full traffic-rule enforcement. TerraZero uses logged data only as a source of real-world map geometry, populating each map with randomized rule-based road users and signal controllers and randomizing agent dynamics, rewards, and sizes per episode, so one map yields an effectively unbounded set of scenarios. Every reported policy trains from scratch by reinforcement learning alone, with zero human demonstrations, no imitation, no logged trajectories, and no fallback planner at inference, on a compute-efficient self-play recipe scaled across GPUs. The policies generalize zero-shot across cities and datasets, including emergent left-hand-traffic driving without explicit supervision. As an ego policy, a single checkpoint is, to our knowledge, the first fully learned policy to top both val14 and the interactive long-tail InterPlan suite. On Waymo Open Sim Agents realism the same recipe outperforms other demonstration-free methods and is competitive with the strongest reference-anchored self-play method. One stack serves both roles: state-of-the-art demonstration-free driving policies across dynamics for cars and trucks, and sim agents that jointly control vehicles, pedestrians, and cyclists.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。