用大模型+预测控制实现自动驾驶安全灵活切换
LVLM-MPC Collaboration for Autonomous Driving: A Safety-Aware and Task-Scalable Control Architecture
- 大模型生成任务指令，自动构建符合安全约束的预测控制
- 系统可验证任务可行性，避免不切实际的驾驶切换
- 适合追求高灵活性又需保障安全的自动驾驶研发者
本文提出一种大型视觉语言模型（LVLM）与模型预测控制（MPC）协同的新框架，实现自动驾驶中任务的可扩展性与安全性。LVLM擅长在多样场景下进行高层任务规划，但其推理与底层运动规划可行性不一致，存在安全隐患和任务切换不畅问题。本文将LVLM与MPC Builder结合，基于LVLM生成的符号化任务指令，动态构建满足最优性与安全性的MPC控制器。生成的MPC可反馈任务可行性信息，支持任务切换的合理执行或拒绝，并生成适应切换的控制策略。该方法实现了安全、灵活、可扩展的控制架构，弥合了前沿基础模型与可靠车辆运行之间的差距。通过仿真实验验证，系统能在高速场景中安全高效执行，同时保持LVLM的灵活性与适应性。
原文摘要 · Abstract (English)
This paper proposes a novel Large Vision-Language Model (LVLM) and Model Predictive Control (MPC) integration framework that delivers both task scalability and safety for Autonomous Driving (AD). LVLMs excel at high-level task planning across diverse driving scenarios. However, since these foundation models are not specifically designed for driving and their reasoning is not consistent with the feasibility of low-level motion planning, concerns remain regarding safety and smooth task switching. This paper integrates LVLMs with MPC Builder, which automatically generates MPCs on demand, based on symbolic task commands generated by the LVLM, while ensuring optimality and safety. The generated MPCs can strongly assist the execution or rejection of LVLM-driven task switching by providing feedback on the feasibility of the given tasks and generating task-switching-aware MPCs. Our approach provides a safe, flexible, and adaptable control framework, bridging the gap between cutting-edge foundation models and reliable vehicle operation. We demonstrate the effectiveness of our approach through a simulation experiment, showing that our system can safely and effectively handle highway driving while maintaining the flexibility and adaptability of LVLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。