arXiv:2411.11683cs.ROcs.AI2024-11被引 19

攻击机器人视觉语言模型供应链,植入隐蔽后门操控机械臂行为

Robot Collapse: Supply Chain Backdoor Attacks Against VLM-based Robotic Manipulation

  • 通过后门关系在模块化策略中嵌入恶意组件,操控大模型感知路径
  • 三种攻击模式实现在18个真实任务中的灵活控制,成功率超90%
  • 适合安全研究者与机器人系统开发者关注供应链防护

机器人操作策略正越来越多地依赖大型语言模型(LLMs)和视觉语言模型(VLMs)的感知与理解能力。尽管推理阶段攻击已受广泛关注,针对机器人策略模型供应链安全的后门攻击仍鲜有研究。为此,本文提出 exttt{TrojanRobot},一种面向模型供应链攻击的后门注入框架,通过在LLM到VLM的路径中嵌入恶意模块实现系统操控。基础设计将该模块设为后门微调的VLM;为进一步提升攻击效果,提出 extit{LVLM-as-a-backdoor} 策略,利用上下文指令学习(ICIL)通过被污染的系统提示引导大视觉语言模型(LVLM)行为。此外,设计了三种主攻攻击:置换、停滞与故意型,可实现灵活后门效应。在18个真实世界操作任务及4种VLM上开展的物理世界与仿真实验验证了 exttt{TrojanRobot}的优越性。

原文摘要 · Abstract (English)

Robotic manipulation policies are increasingly empowered by \textit{large language models} (LLMs) and \textit{vision-language models} (VLMs), leveraging their understanding and perception capabilities. Recently, inference-time attacks against robotic manipulation have been extensively studied, yet backdoor attacks targeting model supply chain security in robotic policies remain largely unexplored. To fill this gap, we propose \texttt{TrojanRobot}, a backdoor injection framework for model supply chain attack scenarios, which embeds a malicious module into modular robotic policies via backdoor relationships to manipulate the LLM-to-VLM pathway and compromise the system. Our vanilla design instantiates this module as a backdoor-finetuned VLM. To further enhance attack performance, we propose a prime scheme by introducing the concept of \textit{LVLM-as-a-backdoor}, which leverages \textit{in-context instruction learning} (ICIL) to steer \textit{large vision-language model} (LVLM) behavior through backdoored system prompts. Moreover, we develop three types of prime attacks, \textit{permutation}, \textit{stagnation}, and \textit{intentional}, achieving flexible backdoor attack effects. Extensive physical-world and simulator experiments on 18 real-world manipulation tasks and 4 VLMs verify the superiority of proposed \texttt{TrojanRobot}

机器人安全后门攻击VLM供应链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。