无需专家数据,通过自对弈训练驾驶策略并对接视觉模型。
TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations

- 用自对弈强化学习训练驾驶策略,解耦感知与决策。
- 在真实3D场景中性能超越或媲美现有端到端方法。
- 仅需图像与场景状态配对数据,无需专家示范轨迹。
端到端自动驾驶已在基准测试和实际部署中达到顶尖水平,但其标准训练流程成本高昂:收集和标注数百万帧驾驶数据代价大,且在图像上进行闭环强化学习受制于逼真渲染的每步开销及大型视觉骨干网络的前向计算。而在向量化模拟器中的自对弈则改变了经济性:每秒可生成数百万步轨迹,状态分布自然包含碰撞、险情和恢复等丰富样本,是传统驾驶日志无法覆盖的。本文方法利用这一不对称性,将驾驶策略学习与感知学习解耦:先通过自对弈预训练单一策略,再通过动作KL散度和批关系低秩结构损失,将其隐空间与预训练视觉主干对齐。动作目标来自自对弈策略,因此对齐过程不依赖于人工标注轨迹——仅需(图像, 场景状态)配对数据即可,无需模仿学习所依赖的精心筛选的专家示范。在基于3D高斯泼溅的逼真闭环场景中,最终得到的端到端策略性能达到或超过先前方法。
原文摘要 · Abstract (English)
End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all stages: collecting and labeling millions of driving frames is costly, and closed-loop RL on images is bottlenecked by the per-step cost of photorealistic rendering plus a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: millions of rollout steps per second, and a state distribution naturally rich in collisions, near-misses, and recoveries that no driving log contains. Our approach exploits this asymmetry by decoupling learning to drive from learning to see. We pretrain a single policy by self-play, then align its latent space with a pretrained vision backbone, through the action KL divergence and a batch-relational low-rank structural loss. The action target comes from the self-play policy, so alignment never supervises against a logged trajectory: a paired dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. On photorealistic 3D Gaussian splatting closed-loop scenarios, the resulting end-to-end policy matches or exceeds prior end-to-end methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。