用短语干扰机器人安全机制,让其无故停机。
Semantic Denial of Service in LLM-controlled robots

- 向机器人音频通道注入1-5词的安全类短语,触发误判停机。
- 攻击成功率高,且防御措施会引发新形式的干扰(如循环确认)。
- 适合关注智能体安全与系统架构设计的研究者阅读。
以安全为导向的指令遵循本应保障大语言模型控制的机器人安全,但我们也发现它同时带来了可用性攻击面。通过在机器人的音频通道中注入简短且看似合理的安全提示(1-5个词),攻击者可触发模型的安全推理机制,导致执行中断或停机,而无需越狱模型或篡改其策略。在具身场景中,这是一种语义拒绝服务攻击:代理因注入信号看似合法警告而停止运行。我们在四种视觉-语言模型、七种提示级防御、三种部署模式以及单次与多次注入设置下进行了测试,发现仅靠提示的防御会牺牲真实危险响应能力来抑制攻击。最强的防御虽能降低部分模型的硬停机成功率,但只是改变了干扰形式而非消除其本质:被抑制的硬停机转为确认循环和虚假警报,我们通过中断成功率(DSR)进行量化评估。此外,我们发现注入内容多样化比重复相同短语更有效,表明模型将多种安全信号视为相互佐证。实际启示在于系统架构层面:直接将未经验证的音频文本输入大模型,会带来本可避免的安全监控与动作选择间的耦合风险。
原文摘要 · Abstract (English)
Safety-oriented instruction-following is supposed to keep LLM-controlled robots safe. We show it also creates an availability attack surface. By injecting short safety-plausible phrases (1-5 tokens) into a robots audio channel, an adversary can trigger the models safety reasoning to halt or disrupt execution without jailbreaking the model or overriding its policy. In the embodied setting, this is a semantic denial-of-service attack: the agent stops because the injected signal looks like a legitimate alert. Across four vision-language models, seven prompt-level defenses, three deployment modes, and single- and multi-injection settings, we find that prompt-only defenses trade off attack suppression against genuine hazard response. The strongest defenses reduce hard-stop attack success on some models, but defenses change the form of disruption, not its fact: suppressed hard stops re-emerge as acknowledge loops and false alerts, which we measure with Disruption Success Rate (DSR). We further find that injection variety is consistently more effective than repeating the same phrase, suggesting that models treat diverse safety cues as corroborating evidence. The practical implication is architectural rather than prompt-level: systems that route unauthenticated audio text directly into the LLM create an avoidable security dependency between safety monitoring and action selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。