用大模型提升自动驾驶轨迹规划,让车辆更懂复杂路况。
Large Language Model Enhanced Differentiable Trajectory Planning for IoT-Enabled Autonomous Driving

- 引入周边车辆轨迹重构作为额外监督信号,增强训练覆盖。
- 在实时约束下实现高阶场景语义提取,性能优于基准83.63分。
- 支持在线优化与梯度回传,适合真实道路部署的闭环系统。
物联网赋能的自动驾驶规划是智能交通系统的关键环节,需在复杂城市环境中基于多源上下文信息生成安全、高效且可执行的轨迹。尽管模仿学习(IL)在大规模数据集上表现良好,但其规划器仍面临长尾复杂交互覆盖不足、下游约束优化一致性弱、实时条件下高阶场景语义利用不充分等问题。为此,本文提出一种大语言模型(LLM)增强的可微分轨迹规划框架。首先设计以周围代理为中心的数据增强策略,将邻近车辆轨迹重组织为额外规划监督信号,从而扩展训练分布而不需采集新原始数据。其次,构建复杂度感知的异步式LLM语义增强模块,在可控在线开销下提取场景相关高阶语义特征。此外,引入可微分优化模块,在显式残差惩罚下优化生成轨迹,并将优化梯度反向传播至上游规划器。实验表明,该方法在nuPlan闭环非反应式与反应式Hard20基准上分别取得83.63和78.29的最优综合得分;CARLA-ROS测试进一步验证了其在线部署与实时闭环执行能力。
原文摘要 · Abstract (English)
Autonomous driving planning is a key component of IoT-enabled intelligent transportation systems, requiring vehicles to generate safe, efficient, and executable trajectories in complex urban environments from multi-source contextual information. While imitation learning (IL) has shown promise on large-scale datasets, IL-based planners still suffer from limited coverage of complex long-tail interactions, weak consistency with downstream constrained refinement, and insufficient use of high level scene semantics under real time constraints. To address these issues, this paper proposes a large language model (LLM) enhanced differentiable trajectory planning framework for IoT-enabled autonomous driving. Specifically, we introduce a surrounding agent centric data augmentation strategy to reorganize sur rounding agent trajectories as additional planning supervision, thereby improving the training distribution without collecting additional raw data. We further design a complexity-aware asyn chronous LLM-based semantic enhancement module to extract scene-related high-level semantic features with controlled online overhead. In addition, a differentiable optimization module is incorporated to refine generated trajectories with explicit residual penalties while backpropagating optimization gradients to the upstream planner. Experiments show that the proposed method achieves the best overall scores of 83.63 and 78.29 on the nuPlan closed-loop nonreactive and reactive Hard20 benchmarks, respectively, and CARLA-ROS tests further verify its online deployment and real time closed-loop execution capability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。