提出可通用的视觉语言动作模型攻击与防御方法。
Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models
- 通过干扰视觉文本语义对齐生成可放置于摄像头视野的对抗补丁。
- 在LIBERO基准上使顶尖模型任务失败率显著上升,攻击成功率超90%。
- 适合关注机器人安全、对抗鲁棒性的研究者和工程师。
视觉-语言-动作(VLA)模型在机器人学习中取得突破性进展,使机器人能根据自然语言指令执行复杂物理任务。然而其对抗鲁棒性仍待深入研究。本文提出一种模型无关的对抗补丁攻击方法——嵌入破坏补丁攻击(EDPA),该方法直接生成可放置于摄像头视野内的补丁。相较于以往方法,EDPA无需预先了解模型架构或控制机械臂信息,即可适用于多种VLA模型。其核心机制为:(i) 扰乱视觉与文本隐空间表示间的语义对齐,(ii) 最大化对抗输入与原始输入在隐空间表示上的差异。通过优化上述目标,EDPA扭曲VLA对视觉信息的理解,导致模型反复生成错误动作,最终无法完成任务。为此,我们提出一种针对视觉编码器的对抗微调策略,使其在干净与对抗扰动输入下产生相似的隐空间表示。在广泛使用的LIBERO机器人仿真基准上的大量实验表明,EDPA显著提升前沿VLA模型的任务失败率,而所提防御策略有效缓解性能下降。代码已公开于 https://edpa-attack.github.io/。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have achieved revolutionary progress in robot learning, enabling robots to execute complex physical robot tasks from natural language instructions. Despite this progress, their adversarial robustness remains underexplored. In this work, we propose both adversarial patch attack and corresponding defense strategies for VLA models. We first introduce the Embedding Disruption Patch Attack (EDPA), a model-agnostic adversarial attack that generates patches directly placeable within the camera's view. In comparison to prior methods, EDPA can be readily applied to different VLA models without requiring prior knowledge of the model architecture, or the controlled robotic manipulator. EDPA constructs these patches by (i) disrupting the semantic alignment between visual and textual latent representations, and (ii) maximizing the discrepancy of latent representations between adversarial and corresponding clean visual inputs. Through the optimization of these objectives, EDPA distorts the VLA's interpretation of visual information, causing the model to repeatedly generate incorrect actions and ultimately result in failure to complete the given robotic task. To counter this, we propose an adversarial fine-tuning scheme for the visual encoder, in which the encoder is optimized to produce similar latent representations for both clean and adversarially perturbed visual inputs. Extensive evaluations on the widely recognized LIBERO robotic simulation benchmark demonstrate that EDPA substantially increases the task failure rate of cutting-edge VLA models, while our proposed defense effectively mitigates this degradation. The codebase is accessible via the homepage at https://edpa-attack.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。