arXiv:2604.01259cs.RO2026-04被引 4

首个面向视觉语言模型的闭环自动驾驶评测基准,可评估模型在异常驾驶场景下的表现。

Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models

  • 构建闭环生成器DriveCommenter,自动生成多样化的驾驶问答对。
  • 支持在CARLA仿真中评估模型在偏离路线等极端情况下的表现。
  • 提供可直接接入现代视觉语言模型的统一评测框架,适合研究者对比新旧方法。

随着视觉语言模型(VLM)在自动驾驶领域的兴起,其应用受到广泛关注。然而,现有VLM4AD评测仍基于开环方式,即通过静态问答数据集评估场景理解能力,无法有效检验模型在分布外状态(如罕见驾驶偏差)下的表现。为此,我们提出Bench2Drive-VL,作为Bench2Drive的扩展,首次将闭环评估引入VLM驱动的自动驾驶。该基准包含:(1) DriveCommenter,一个自动为CARLA仿真中所有驾驶情境生成行为相关问答对的闭环生成器,涵盖严重偏离路径与非道路状态;(2) 统一协议与接口,支持现代VLM直接接入闭环环境,与传统智能体进行对比;(3) 可配置的图结构思维链推理与控制框架,兼容多格式视觉输入;(4) 完整开发生态。所有代码与标注数据集已开源。

原文摘要 · Abstract (English)

With the rise of vision-language models (VLM), their application for autonomous driving (VLM4AD) has gained significant attention. Meanwhile, in autonomous driving, closed-loop evaluation has become widely recognized as a more reliable validation method than open-loop evaluation, as it can evaluate the performance of the model under cumulative errors and out-of-distribution inputs. However, existing VLM4AD benchmarks evaluate the model`s scene understanding ability under open-loop, i.e., via static question-answer (QA) dataset. This kind of evaluation fails to assess the VLMs performance under out-of-distribution states rarely appeared in the human collected datasets.To this end, we present Bench2Drive-VL, an extension of Bench2Drive that brings closed-loop evaluation to VLM-based driving, which introduces: (1) DriveCommenter, a closed-loop generator that automatically generates diverse, behavior-grounded question-answer pairs for all driving situations in CARLA,including severe off-route and off-road deviations previously unassessable in simulation. (2) A unified protocol and interface that allows modern VLMs to be directly plugged into the Bench2Drive closed-loop environment to compare with traditional agents. (3) A flexible reasoning and control framework, supporting multi-format visual inputs and configurable graph-based chain-of-thought execution. (4) A complete development ecosystem. Together, these components form a comprehensive closed-loop benchmark for VLM4AD. All codes and annotated datasets are open sourced.

自动驾驶视觉语言模型闭环评测CARLA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。