首次系统研究VLA模型在物理传感器攻击下的脆弱性,提出防御方案。
Phantom Menace: Exploring and Enhancing the Robustness of VLA Models Against Physical Sensor Attacks
- 构建'真实-模拟-真实'框架,自动生成相机与麦克风的物理攻击
- 实测发现不同任务和模型设计下存在显著攻击敏感性差异
- 基于对抗训练的防御可提升鲁棒性,同时保持原模型性能
视觉-语言-动作(VLA)模型通过整合摄像头视觉信号与麦克风音频信号等多模态数据,实现从感知到动作的端到端控制,显著提升机器人系统能力。然而,此类系统高度依赖传感器输入,其在真实世界中面对物理传感器攻击的安全性仍严重缺乏研究。本文首次系统开展针对VLA模型的物理传感器攻击研究,量化攻击影响并探索防御策略。我们提出一种新型'真实-模拟-真实'框架,可自动模拟六类摄像头攻击和两类麦克风攻击,均在真实机器人上验证。大规模实验覆盖多种VLA架构与任务,在不同攻击参数下揭示了显著脆弱性,且敏感度与任务类型及模型结构密切相关。进一步提出基于对抗训练的防御方法,有效提升模型对分布外物理扰动的鲁棒性,同时维持原有性能。研究结果凸显了在安全关键场景中建立标准化鲁棒性评估基准与防护机制的紧迫性。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models revolutionize robotic systems by enabling end-to-end perception-to-action pipelines that integrate multiple sensory modalities, such as visual signals processed by cameras and auditory signals captured by microphones. This multi-modality integration allows VLA models to interpret complex, real-world environments using diverse sensor data streams. Given the fact that VLA-based systems heavily rely on the sensory input, the security of VLA models against physical-world sensor attacks remains critically underexplored. To address this gap, we present the first systematic study of physical sensor attacks against VLAs, quantifying the influence of sensor attacks and investigating the defenses for VLA models. We introduce a novel "Real-Sim-Real" framework that automatically simulates physics-based sensor attack vectors, including six attacks targeting cameras and two targeting microphones, and validates them on real robotic systems. Through large-scale evaluations across various VLA architectures and tasks under varying attack parameters, we demonstrate significant vulnerabilities, with susceptibility patterns that reveal critical dependencies on task types and model designs. We further develop an adversarial-training-based defense that enhances VLA robustness against out-of-distribution physical perturbations caused by sensor attacks while preserving model performance. Our findings expose an urgent need for standardized robustness benchmarks and mitigation strategies to secure VLA deployments in safety-critical environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。