视觉语言模型让自动驾驶感知更抗攻击，无需额外训练即可保持高鲁棒性。
Toward Inherently Robust VLMs Against Visual Perception Attacks
- 用专为汽车感知优化的视觉语言模型，实现无需对抗训练的内在抗攻击能力。
- 在未知攻击下，传统模型准确率下降33%-74%，而新模型仅降8%以内。
- 单模型多任务部署（Tandem）兼顾效率与鲁棒性，适合实际系统集成。
自动驾驶车辆依赖深度神经网络（DNN）进行交通标志识别、车道居中和车辆检测，但这些模型易受攻击导致误分类，威胁安全。现有防御方法（如对抗训练）常泛化能力差且损害正常精度。本文提出专用于自动驾驶感知的视觉-语言模型（V2LMs），并证明其无需对抗训练即可具备更强的内在鲁棒性，对抗攻击下的准确率显著高于传统DNN。研究了两种部署方式：Solo（针对各任务分别设计的V2LM）和Tandem（单一V2LM完成三项任务）。在攻击下，DNN准确率下降33%-74%，而V2LM平均下降不足8%。Tandem在保持与Solo相当鲁棒性的同时，内存效率更高。此外，还探索了将V2LMs并行集成到现有感知系统以增强韧性。结果表明，V2LMs是实现安全、稳健自动驾驶感知的有前景方向。
原文摘要 · Abstract (English)
Autonomous vehicles rely on deep neural networks (DNNs) for traffic sign recognition, lane centering, and vehicle detection, yet these models are vulnerable to attacks that induce misclassification and threaten safety. Existing defenses (e.g., adversarial training) often fail to generalize and degrade clean accuracy. We introduce Vehicle Vision-Language Models (V2LMs), fine-tuned vision-language models specialized for autonomous vehicle perception, and show that they are inherently more robust to unseen attacks without adversarial training, maintaining substantially higher adversarial accuracy than conventional DNNs. We study two deployments: Solo (task-specific V2LMs) and Tandem (a single V2LM for all three tasks). Under attacks, DNNs drop 33-74%, whereas V2LMs decline by under 8% on average. Tandem achieves comparable robustness to Solo while being more memory-efficient. We also explore integrating V2LMs in parallel with existing perception stacks to enhance resilience. Our results suggest V2LMs are a promising path toward secure, robust AV perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。