攻击视觉语言模型GUI代理的响应速度,制造隐蔽延迟漏洞。
SlowBA: An efficiency backdoor attack towards VLM-based GUI agents
- 用触发模式诱导长推理链,操控响应时延。
- 在多数据集上使响应长度增加超3倍,延迟显著上升。
- 基于弹窗触发,隐蔽性强,适合安全研究者关注。
基于视觉语言模型(VLM)的图形用户界面(GUI)代理不仅需准确执行操作,还需低延迟响应用户指令。现有研究主要关注动作正确性,而对响应效率的安全风险尚未深入探讨。本文提出SlowBA,一种针对VLM-based GUI代理响应速度的新型后门攻击。核心思路是通过特定触发模式诱导过长的推理链,从而操纵响应延迟。为此,我们设计两阶段奖励级后门注入(RBI)策略:首先对齐长响应格式,再通过强化学习学习触发感知激活机制。此外,我们采用真实场景中的弹窗窗口作为触发器,提升攻击隐蔽性。在多个数据集和基线上的实验表明,SlowBA能显著增加响应长度与延迟,同时保持任务准确性。该攻击在低中毒比例下仍有效,并能绕过多种防御机制。这些发现揭示了GUI代理中被忽视的安全漏洞,凸显了兼顾动作正确性与响应效率防护的必要性。代码已开源:https://github.com/tu-tuing/SlowBA。
原文摘要 · Abstract (English)
Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low latency. While existing research on GUI-agent security mainly focuses on manipulating action correctness, the security risks related to response efficiency remain largely unexplored. In this paper, we introduce SlowBA, a novel backdoor attack that targets the responsiveness of VLM-based GUI agents. The key idea is to manipulate response latency by inducing excessively long reasoning chains under specific trigger patterns. To achieve this, we propose a two-stage reward-level backdoor injection (RBI) strategy that first aligns the long-response format and then learns trigger-aware activation through reinforcement learning. In addition, we design realistic pop-up windows as triggers that naturally appear in GUI environments, improving the stealthiness of the attack. Extensive experiments across multiple datasets and baselines demonstrate that SlowBA can significantly increase response length and latency while largely preserving task accuracy. The attack remains effective even with a small poisoning ratio and under several defense settings. These findings reveal a previously overlooked security vulnerability in GUI agents and highlight the need for defenses that consider both action correctness and response efficiency. Code can be found in https://github.com/tu-tuing/SlowBA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。