提出真实动态环境下的GUI代理攻击框架,揭露现有模型脆弱性。
Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments
- 用LLM生成多样化网页模拟数据,提升攻击真实性
- 通过注意力黑洞机制优化触发器,提升攻击效果
- 在6个真实网站上验证,显著超越现有方法
图形用户界面(GUI)代理被广泛用于与在线网页服务交互,但其暴露于开放世界内容中,易受环境注入攻击(EIA)影响。此类攻击中,攻击者可向网页注入精心设计的触发器,操控其他用户的GUI代理行为。本文指出,现有EIA研究缺乏现实性,未能捕捉真实网络内容的动态特性,常假设触发器的屏幕位置和视觉上下文在训练与测试阶段保持一致。为此,我们提出一个更贴近实际的动态环境威胁模型:攻击者为普通用户,触发器嵌入动态变化的环境中。在此模型下,现有方法表现大幅下降,表明其对GUI代理漏洞的暴露程度被严重高估。为有效揭示现有代理的隐藏缺陷,我们提出Chameleon攻击框架,包含两项创新:(1) 引入基于LLM的环境仿真,自动生成多样且高保真的网页模拟,以模拟真实世界动态环境的变异性;(2) 提出注意力黑洞机制,将注意力权重转化为显式监督信号,促使代理对无关上下文保持不敏感,从而提升在动态环境中的鲁棒性。我们在六个真实网站和四个代表性LVLM驱动的GUI代理上评估了Chameleon,结果显著优于现有方法。
原文摘要 · Abstract (English)
Graphical User Interface (GUI) agents are increasingly deployed to interact with online web services, yet their exposure to open-world content renders them vulnerable to Environmental Injection Attacks (EIAs). In these attacks, an attacker can inject crafted triggers into website to manipulate the behavior of GUI agents used by other users. In this paper, we find that most existing EIA studies fall short of realism. In particular, they fail to capture the dynamic nature of real-world web content, often assuming that a trigger's on-screen position and surrounding visual context remain largely consistent between training and testing. To better reflect practice, we introduce a realistic dynamic-environment threat model in which the attacker is a regular user and the trigger is embedded within a dynamically changing environment. Under this threat model, existing approaches largely fail, suggesting that their effectiveness in exposing GUI agent vulnerabilities has been substantially overestimated. To expose the hidden vulnerabilities of existing GUI agents effectively, we propose Chameleon, an attack framework with two key novelties designed for dynamic environments. (1) To synthesize more realistic training data, we introduce LLM-Driven Environment Simulation, which automatically generates diverse, high-fidelity webpage simulations that mimic the variability of real-world dynamic environments. (2) To optimize the trigger more effectively, we introduce Attention Black Hole, which converts attention weights into explicit supervisory signals. This mechanism encourages the agent to remain insensitive to irrelevant surrounding content, thereby improving robustness in dynamic environments. We evaluate Chameleon on six realistic websites and four representative LVLM-powered GUI agents, where it significantly outperforms existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。