用像素自对弈训练端到端自动驾驶,提升真实场景适应力。
Scaling Self-Play for End-to-End Driving

- 在像素空间中进行大规模自对弈,无需人工轨迹监督。
- 每秒处理5万智能体步数,支持高效训练。
- 适合追求高鲁棒性自动驾驶系统的研发团队。
端到端自动驾驶模型通常依赖离线人类示范数据训练,这类数据覆盖状态有限且缺乏闭环反馈,导致部署时误差累积严重,对长尾交互场景脆弱。为克服此问题,本文提出一种新策略:在仿真环境中直接从像素出发进行大规模自对弈训练。现有自对弈方法多依赖向量化的鸟瞰图观测,与直接处理传感器输入的端到端策略不兼容。为此,我们设计了Gigapixel——一个高吞吐量批量驾驶模拟器,支持透视渲染,实现像素级自对弈。该模拟器采用简化的边界框世界,保留关键场景结构,达到每秒50,000智能体步的吞吐率。由于像素空间自对弈强化学习在大模型规模下样本效率极低,我们提出自对弈DAgger训练:通过特权强化学习教师对策略进行在线策略蒸馏。为弥合仿真与真实差距,后续引入轻量感知适配,将自对弈训练策略迁移到真实传感器数据。在HUGSIM和NAVSIM-v2基准上,该方法无需人类轨迹监督即取得竞争力表现。进一步验证显示,扩大自对弈训练规模可带来性能持续提升,确立自对弈作为端到端模型训练的可行且可扩展路径。
原文摘要 · Abstract (English)
End-to-end autonomous driving models are typically trained on offline human-demonstration datasets that provide limited state coverage and often no closed-loop feedback, making them prone to compounding errors when deployed in closed-loop and brittle to long-tail agent interactions. To overcome these limitations, we propose an alternative strategy for training end-to-end driving models: large-scale self-play directly from pixels in simulation. While prior self-play approaches have shown promising transfer to real-world driving, they typically assume vectorized Bird's-Eye-View (BEV) observations that are incompatible with end-to-end policies operating directly on sensor observations. To this end, we introduce Gigapixel, a high-throughput batched driving simulator with perspective rendering, enabling scalable self-play directly from pixel observations. Rather than targeting compute-costly photorealistic sensor simulation, Gigapixel renders a simplified bounding-box world that preserves essential scene structure while achieving throughput at 50k agent steps per second. Since direct pixel-space self-play RL is prohibitively sample-inefficient at end-to-end model scale, we propose self-play DAgger training: we train pixel-based policies in self-play via on-policy distillation from a privileged RL teacher. To bridge the sim-to-real gap, we subsequently transfer the self-play trained policies to real-world sensor data through lightweight perception adaptation. Policies trained in Gigapixel and adapted to real-world sensor data achieve competitive performance on the HUGSIM and NAVSIM-v2 benchmarks without human trajectory supervision. Moreover, scaling self-play training yields proportional gains in policy performance, establishing self-play as a practical and scalable strategy for training end-to-end models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。