arXiv:2607.05783cs.CV2026-07TPAMI

首次系统评估驾驶环境幻觉对车道感知的影响,揭示安全隐患并提出防御方案。

Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective

论文配图:Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective
图 1 · 摘自论文原文
  • 构建首个针对环境幻觉的车道感知评测基准LanEvil++
  • 阴影导致检测准确率最高下降7.20%,两类模型性能均显著下降
  • 提出多模态防御方法MIDA,提升鲁棒性4%以上,适合自动驾驶安全研究

环境幻觉(如阴影、反光和轮胎痕迹)是真实道路环境中自然存在却常被忽视的现象,可能干扰视觉感知,导致场景误判,对自动驾驶系统构成严重安全风险。现有研究大多忽略此类现象,形成关键空白。本文从车道感知视角出发,研究自动驾驶系统的鲁棒性,聚焦传统车道检测(LD)与基于视觉-语言模型的系统(ADVLMs)。提出首个评测基准LanEvil++,包含14类幻觉,利用CARLA模拟器生成94个高保真、可完全控制的3D场景,构建含90,292张标注图像、1,596段视频及41,855个视觉问答对的数据集。大量实验表明,环境幻觉显著降低先进LD模型性能:平均准确率下降5.27%,F1分数下降10.49%;ADVLMs的GPT分数下降2.03%,Language分数下降0.75%。其中阴影为最严重干扰因素,使准确率最高下降7.20%。闭环仿真显示幻觉可引发错误驾驶决策,真实案例亦证实其在实际交通中导致安全危机。为此,提出多模态幻觉防御方法MIDA,有效提升鲁棒性:在复杂条件下使LD模型鲁棒性提升4.23%,ADVLMs提升3.82%。

原文摘要 · Abstract (English)

Environmental illusions (eg., shadows, reflections, and tire marks) are naturally existing yet overlooked phenomena in real-world driving environments. They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving (AD) systems. However, existing researches largely overlook these phenomena, leaving a critical gap. To address this issue, we study AD robustness through the lane perception perspective, a fundamental task supporting core functions like cruise control and lane centering. We focus on two representative models: conventional lane detection (LD) and vision-language model-based systems (ADVLMs). In this work, we introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions. LanEvil++ encompasses 14 types of illusions and leverages the CARLA simulator to generate 94 high-fidelity, fully controllable 3D scenes, yielding a dataset of 90,292 annotated images, 1,596 video clips, and 41,855 visual question answering pairs. Extensive evaluations demonstrate that environmental illusions substantially degrade the performance of state-of-the-art LD methods. On average, LD models experience a 5.27% drop in Accuracy and a 10.49% decline in F1-score, while ADVLMs show a 2.03% reduction in GPT-score and a 0.75% drop in Language-score. Among all illusions, shadows emerge as the most disruptive factor, reducing accuracy by up to 7.20%. Furthermore, closed-loop simulations reveal that these illusions can lead to incorrect driving decisions. Complementary real-world case studies highlight safety-critical failures in actual traffic scenes. To enhance robustness, we propose the Multimodal Illusion Defense Approach (MIDA). MIDA achieves substantial gains under challenging conditions, boosting robustness by 4.23% on LD models and 3.82% on ADVLMs.

自动驾驶车道检测鲁棒性幻觉防御

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。