攻击智能机器人视觉指令,用假路牌骗它执行恶意操作
CHAI: Command Hijacking against embodied AI
- 在真实环境里藏虚假文字指令,诱骗机器人理解错误
- 对四种机器人系统攻击成功率达90%以上,超越现有方法
- 适合研究智能体安全与对抗攻击的学者参考
具身人工智能(Embodied AI)通过感知与行动结合的常识推理,能在数据稀缺场景下泛化并适应新环境,但其多模态语言理解能力也带来新型安全风险。本文提出CHAI(命令劫持针对具身AI),一种物理环境中的间接提示注入攻击,利用AI模型对多模态输入的语义理解能力,在视觉输入中嵌入欺骗性自然语言指令(如误导性标识),系统搜索词元空间,构建提示词典,并引导攻击模型生成视觉攻击提示。我们在四种大型视觉语言模型(LVLM)代理上评估:无人机紧急降落、自动驾驶、空中目标追踪及真实机器人车辆。实验表明,CHAI持续优于当前最先进攻击方法。该攻击利用下一代具身AI系统的语义与多模态推理优势,凸显了亟需突破传统对抗鲁棒性的防御体系。
原文摘要 · Abstract (English)
Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a physical environment indirect prompt injection attack that exploits the multimodal language interpretation abilities of AI models. CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents: drone emergency landing, autonomous driving, aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。