无需人类示范，纯靠规则奖励训练端到端自动驾驶系统。
Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer
- 用可枚举轨迹动作的强化学习优化算法，实现无示范训练。
- 在长尾场景和封闭环路测试中性能超越模仿学习方法。
- 适合研究纯强化学习自动驾驶或数据稀缺场景应用者。
人类示范是端到端自动驾驶的核心，但在长尾和高危场景中极为稀缺。当前多数方法仍依赖人类示范：模仿学习（IL）直接使用示范，强化学习（RL）虽有潜力降低依赖，但多数仍隐性依赖示范。纯基于奖励的强化学习可摆脱示范依赖，但传统策略梯度方法存在冷启动问题。本文提出ZTRS（零人类示范端到端自动驾驶与轨迹评分器），一种完全基于真实图像与规则奖励训练的端到端规划范式，不依赖任何人类示范。通过提出的全面策略优化（EPO），一种针对可枚举轨迹动作和密集监督设计的策略梯度变体，ZTRS显著提升模型对长尾驾驶场景的泛化能力。我们在长尾导航数据集Navhard和封闭环路仿真数据集HUGSIM上均取得优于现有模仿学习方法的最先进性能。
原文摘要 · Abstract (English)
Human demonstrations are widely considered the cornerstone of end-to-end (E2E) autonomous driving despite human demonstration's scarcity for long-tail and safety-critical scenarios. Nonetheless, current E2E autonomous driving (AD) training paradigms continue to rely on human demonstrations. Imitation learning (IL) requires human demonstrations for training, whereas reinforcement learning (RL) has emerged as a promising alternative to reduce this dependency. However, most existing RL methods for E2E AD still rely implicitly on human demonstrations. A pure rewards-based RL method can overcome the need for human demonstrations, but general RL policy gradient methods suffer from the cold-start problem. In this paper, we propose ZTRS (Zero-human demonstration end-to-end autonomous driving with TRajectory Scorer) - a complete RL-based E2E planning paradigm trained solely on real-world images and rule-based rewards, entirely without human demonstration. Through our proposed Exhaustive Policy Optimization (EPO), a policy gradient variant tailored for enumerable trajectory actions and dense supervision, ZTRS enables the model to generalize better to long-tail driving scenarios. We demonstrate this generalization through our SOTA performance against IL approaches on both long-tail Navhard and closed-loop HUGSIM datasets. Project page: https://zhenxinli.net/ZTRS/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。