用视觉语言模型增强自动驾驶路径规划,提升复杂交通下的安全性。
VisioPath: Vision-Language Enhanced Model Predictive Control for Safe Autonomous Navigation in Mixed Traffic
- 结合视觉语言模型与模型预测控制,从鸟瞰视频中提取车辆位置、尺寸和速度。
- 构建椭圆碰撞避让势场,在有限时域内优化轨迹并实时调整。
- 适合研究智能驾驶安全决策的学者,尤其关注感知-控制融合系统者。
本文提出VisioPath,一种将视觉语言模型(VLM)与模型预测控制(MPC)结合的新框架,用于动态交通环境中的安全自动驾驶。该方法采用鸟瞰视频处理流程与零样本VLM能力,获取周围车辆的位置、尺寸和速度等结构化信息。基于此感知输出,构建围绕其他交通参与者的椭圆碰撞避让势场,并无缝集成到有限时域最优控制问题中进行轨迹规划。轨迹优化通过带自适应正则化的微分动态规划求解,并嵌入事件触发式MPC循环。为确保无碰撞运动,框架引入安全验证层,评估潜在危险轨迹。在Simulation of Urban Mobility(SUMO)中的大量仿真表明,VisioPath在多个指标上优于传统MPC基线。通过融合现代AI感知与严谨最优控制基础,VisioPath代表了复杂交通系统中安全轨迹规划的重要进展。
原文摘要 · Abstract (English)
In this paper, we introduce VisioPath, a novel framework combining vision-language models (VLMs) with model predictive control (MPC) to enable safe autonomous driving in dynamic traffic environments. The proposed approach leverages a bird's-eye view video processing pipeline and zero-shot VLM capabilities to obtain structured information about surrounding vehicles, including their positions, dimensions, and velocities. Using this rich perception output, we construct elliptical collision-avoidance potential fields around other traffic participants, which are seamlessly integrated into a finite-horizon optimal control problem for trajectory planning. The resulting trajectory optimization is solved via differential dynamic programming with an adaptive regularization scheme and is embedded in an event-triggered MPC loop. To ensure collision-free motion, a safety verification layer is incorporated in the framework that provides an assessment of potential unsafe trajectories. Extensive simulations in Simulation of Urban Mobility (SUMO) demonstrate that VisioPath outperforms conventional MPC baselines across multiple metrics. By combining modern AI-driven perception with the rigorous foundation of optimal control, VisioPath represents a significant step forward in safe trajectory planning for complex traffic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。