用双阶段大模型规划安全驾驶轨迹,融合物理规律与交通知识。
Planning Safety Trajectories with Dual-Phase, Physics-Informed, and Transportation Knowledge-Driven Large Language Models
- 双阶段设计:先反思存记忆,再快速调用相似经验
- 在HighD数据集上五项安全指标全面优于基线模型
- 结合物理动力学与交通规则,减少幻觉和推理延迟
基础模型在驾驶任务中展现强大推理与泛化能力,但在幻觉、不确定性及长推理延迟方面仍存挑战。现有模型虽有避障常识,却缺乏交通领域安全知识。为此,我们提出LetsPi——一种融合物理规律、双阶段架构与交通知识的框架,实现安全且类人轨迹规划。该混合框架将大语言模型(LLM)推理与物理驱动的社会力模型结合:LLM分析场景与历史信息,输出参数与目标点,社会力模型据此生成未来轨迹。双阶段结构通过记忆收集与快速推理平衡效率与质量。记忆收集阶段利用物理感知的LLM进行推理、反思与记忆模块处理,将高质量安全驾驶经验存入记忆库;引入代理安全度量与物理提示技术,增强模型对交通安全部位与物理作用力的理解。快速推理阶段则从记忆库提取相似经验作为少样本示例,简化输入输出,实现无需牺牲安全性的快速规划。在HighD数据集上的大量实验表明,LetsPi在五项安全指标上均超越基线模型。
原文摘要 · Abstract (English)
Foundation models have demonstrated strong reasoning and generalization capabilities in driving-related tasks, including scene understanding, planning, and control. However, they still face challenges in hallucinations, uncertainty, and long inference latency. While existing foundation models have general knowledge of avoiding collisions, they often lack transportation-specific safety knowledge. To overcome these limitations, we introduce LetsPi, a physics-informed, dual-phase, knowledge-driven framework for safe, human-like trajectory planning. To prevent hallucinations and minimize uncertainty, this hybrid framework integrates Large Language Model (LLM) reasoning with physics-informed social force dynamics. LetsPi leverages the LLM to analyze driving scenes and historical information, providing appropriate parameters and target destinations (goals) for the social force model, which then generates the future trajectory. Moreover, the dual-phase architecture balances reasoning and computational efficiency through its Memory Collection phase and Fast Inference phase. The Memory Collection phase leverages the physics-informed LLM to process and refine planning results through reasoning, reflection, and memory modules, storing safe, high-quality driving experiences in a memory bank. Surrogate safety measures and physics-informed prompt techniques are introduced to enhance the LLM's knowledge of transportation safety and physical force, respectively. The Fast Inference phase extracts similar driving experiences as few-shot examples for new scenarios, while simplifying input-output requirements to enable rapid trajectory planning without compromising safety. Extensive experiments using the HighD dataset demonstrate that LetsPi outperforms baseline models across five safety metrics.See PDF for project Github link.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。