arXiv:2607.05180cs.ROcs.CV2026-07

用视觉语言模型预判路况,动态调整自动驾驶安全边界。

VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving

论文配图:VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving
图 1 · 摘自论文原文
  • 用微调的视觉语言模型分析路况,生成自适应安全包络。
  • 在多种天气下完成所有测试,性能优于传统方法。
  • 可适配任意控制器,适合追求高安全性的自动驾驶系统。

恶劣天气等不利驾驶条件同时削弱车辆感知能力与物理执行能力,成为自动驾驶的主要障碍。人类驾驶员通过预判提前调整速度、跟车距离和转向,而现有系统多为事后反应。本文提出VLM-CASE框架,使自动驾驶车辆具备前瞻性能力,同时始终满足形式化安全约束。通过微调的视觉语言模型(VLM,采用低秩适配LoRA)分析前视摄像头图像,判断路面状况与能见度,据此参数化一个基于物理极限和责任敏感安全(RSS)保证的上下文自适应安全包络(CASE),该包络通过共享摩擦预算耦合制动与转向。模型预测控制器(MPC)在包络内自由行驶,而VLM异步运行,不阻塞实时控制。在CARLA仿真中,针对需兼顾横向与纵向控制的任务,在多种天气、路面与光照条件下验证了该框架。结果表明,VLM-CASE-MPC成功完成全部实验,优于传统MPC基线和当前最优的VLM集成控制器。消融实验确认增益源于上下文适应,摩擦与能见度适应具有互补性。该框架对控制器无依赖,可与几乎所有底层控制器搭配,为安全自动驾驶提供新方向。数据集与补充材料见https://github.com/ytj254/VLM-CASE。

原文摘要 · Abstract (English)

Adverse driving conditions, such as bad weather, remain a principal barrier to autonomous driving because they degrade two things at once: what the vehicle can perceive and what it can physically do. Human drivers cope by anticipation, reasoning about the scene and re-budgeting speed, following distance, and steering before grip or sight is lost, whereas current autonomous driving systems at best react after the fact. This paper proposes VLM-CASE, a framework that gives an autonomous vehicle this anticipatory capacity while keeping its motion bounded by a formal safety model at all times. A vision-language model (VLM), fine-tuned with low-rank adaptation (LoRA), reasons about the scene from the front-camera image and reports the road surface and visibility conditions. This output parametrizes a context-adaptive safety envelope (CASE), derived from physical limits and the guarantees of responsibility-sensitive safety, that couples braking and steering through a shared friction budget. A model predictive controller then drives freely within the envelope, while the VLM runs asynchronously so it never blocks the real-time control loop. We validate the framework in closed-loop CARLA simulation on tasks that demand both lateral and longitudinal control, across a range of weather, road-surface, and lighting conditions. The resulting controller, VLM-CASE-MPC, completes all trials, outperforming a conventional MPC baseline and a state-of-the-art VLM-integrated controller. Ablations confirm that the gains come from context adaptation, with the friction and visibility adaptations proving complementary. Furthermore, the framework is controller-agnostic and pairs with almost any low-level controller, offering a promising direction for safe autonomous driving. The dataset and supplementary materials for VLM-CASE are available at https://github.com/ytj254/VLM-CASE.

自动驾驶视觉语言模型安全包络预测控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。