用语言模型思路做自动驾驶路径规划,更安全更靠谱。
Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling
- 分两阶段训练:先学人类驾驶行为,再用规则奖励优化安全与合规
- 在nuPlan数据集上显著提升安全性,尤其在突发反应场景表现最优
- 新算法避免安全问题被稀释,适合追求高可靠性的自动驾驶研发
安全可行的路径规划对真实世界自动驾驶系统至关重要。现有基于学习的规划器严重依赖专家示范数据,不仅缺乏显式安全意识,还可能继承超速等不良驾驶行为。受大语言模型启发,我们提出Plan-R1,一种两阶段轨迹规划框架,将原则对齐与行为学习解耦。第一阶段在专家数据上预训练通用轨迹预测器,捕捉多样化的类人驾驶行为;第二阶段通过基于规则的奖励,使用组相对策略优化(GRPO)对模型进行微调,显式对齐安全、舒适性及交通规则遵守等原则。该框架在保留类人行为的同时增强安全意识并剔除示范中的不良模式。此外,我们发现直接应用GRPO于规划存在关键缺陷:组内归一化会抹除跨组奖励尺度差异,导致罕见但高方差的安全违规组与常见低方差安全组获得相似优势,从而抑制对安全关键目标的优化。为此,我们提出方差解耦的GRPO(VD-GRPO),以中心化和固定缩放替代归一化,保留绝对奖励幅度,确保安全目标在整个训练中保持主导地位。在nuPlan基准上的实验表明,Plan-R1显著提升规划安全性和可行性,达到当前最佳性能,尤其在真实感反应式场景中表现突出。代码已开源:https://github.com/XiaolongTang23/Plan-R1。
原文摘要 · Abstract (English)
Safe and feasible trajectory planning is critical for real-world autonomous driving systems. However, existing learning-based planners rely heavily on expert demonstrations, which not only lack explicit safety awareness but also risk inheriting undesirable behaviors such as speeding from suboptimal human driving data. Inspired by the success of large language models, we propose Plan-R1, a two-stage trajectory planning framework that decouples principle alignment from behavior learning. In the first stage, a general trajectory predictor is pre-trained on expert data to capture diverse, human-like driving behaviors. In the second stage, the model is fine-tuned with rule-based rewards using Group Relative Policy Optimization (GRPO), explicitly aligning ego planning with principles such as safety, comfort, and traffic rule compliance. This two-stage paradigm retains human-like behaviors while enhancing safety awareness and discarding undesirable patterns from demonstrations. Furthermore, we identify a key limitation of directly applying GRPO to planning: group-wise normalization erases cross-group scale differences, causing rare, high-variance safety-violation groups to have similar advantages as abundant low-variance safe groups, thereby suppressing optimization for safety-critical objectives. To address this, we propose Variance-Decoupled GRPO (VD-GRPO), which replaces normalization with centering and fixed scaling to preserve absolute reward magnitudes, ensuring that safety-critical objectives remain dominant throughout training. Experiments on the nuPlan benchmark demonstrate that Plan-R1 significantly improves planning safety and feasibility, achieving state-of-the-art performance, particularly in realistic reactive settings. Our code is available at https://github.com/XiaolongTang23/Plan-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。