arXiv:2607.11560cs.CVcs.AI2026-07

评测自动驾驶视觉语言模型的对抗攻击,发现多视角协同更有效。

Technical Report on the CVPR 2026@AdvML Workshop Challenge

论文配图:Technical Report on the CVPR 2026@AdvML Workshop Challenge
图 1 · 摘自论文原文
  • 用图像和文本扰动攻击多视角驾驶问答模型,保持图像清晰度。
  • 多视角联合优化比单视角攻击效果更好,文本成本越低越优。
  • 字体信息成突破口,适合研究模型安全与防御的团队参考。

视觉语言代理(VLAs)被广泛用于解析复杂驾驶场景并支持关键安全推理。本报告介绍CVPR 2026@AdvML研讨会关于对抗性多模态攻击自动驾驶VLAs的挑战赛。基于DriveLM风格的多视角视觉问答任务,挑战赛以六路同步摄像头图像及结构化驾驶相关问答对表征每个场景。参赛者需生成对抗图像与仅限后缀的文本扰动,使模型输出偏离参考答案,同时保持图像保真度并控制文本代价。竞赛分为两阶段,第二阶段引入隐藏黑盒模型以评估攻击迁移能力。我们阐述任务设计、提交规则、评估协议与排行榜结果,并分析五份有技术报告的领先方案。这些报告揭示若干共性:图像侧攻击受文本惩罚偏好;场景级多视图联合优化优于孤立处理;问答类型与图结构可作为预算分配先验;特征空间目标提升黑盒迁移性;相机图像中的字体内容暴露了持续存在的漏洞。这些发现为未来多模态自动驾驶系统的鲁棒性评估与防御设计提供实践参考。

原文摘要 · Abstract (English)

Vision-language agents (VLAs) are increasingly used to interpret complex driving scenes and support safety-critical reasoning. This report presents the CVPR 2026@AdvML Workshop Challenge on adversarial multimodal attacks against autonomous-driving VLAs. Built on DriveLM-style multi-view visual question answering, the challenge represents each scene with six synchronized camera images and a structured collection of driving-related question-answer pairs. Participants generate adversarial images and suffix-only textual perturbations that induce model responses to deviate from reference answers while preserving image fidelity and limiting textual cost. The competition comprises two phases, with Phase II adding a hidden black-box model to assess transferability. We describe the task design, submission rules, evaluation protocol, and leaderboard results, and then examine five leading submissions for which technical reports were available. Across these reports, several recurring patterns emerge: image-side attacks are favored by the suffix penalty; scene-level, multi-view optimization is more effective than treating views in isolation; QA types and graph structure provide useful priors for allocating attack budget; feature-space objectives can improve black-box transfer; and typographic content embedded in camera images exposes a persistent vulnerability in driving VLAs. These findings provide a practical reference for future robustness evaluation and defense design in multimodal autonomous-driving systems.

对抗攻击自动驾驶多模态视觉语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。