用通知栏里的正常图标做后门,远程操控手机自动操作
AgentRAE: Remote Action Execution through Notification-based Visual Backdoors against Screenshots-based Mobile GUI Agents
- 用对比学习提升代理对图标细微差异的敏感度
- 90%以上成功率实现10种手机操作的远程劫持
- 触发器像普通图标,能绕过主流防御机制
移动图形界面(GUI)代理快速普及,可自主控制应用与操作系统,带来新的系统级攻击面。现有针对网页GUI代理和通用生成式AI模型的后门攻击依赖环境注入或欺骗弹窗,但在基于截图的移动GUI代理上失效,原因包括触发设计空间受限、系统后台干扰及多触发-动作映射冲突。本文提出AgentRAE,一种新型后门攻击,利用视觉自然的触发器(如通知栏中的正常应用图标)实现远程操作执行。为解决自然触发器导致的欠拟合问题并实现精准多目标动作重定向,设计了两阶段流程:首先通过对比学习增强代理对微小图示差异的敏感性;随后通过后门微调将每个触发器与特定移动GUI代理动作关联。大规模评估显示,该后门在保持正常性能的同时,对十种移动操作的攻击成功率超过90%。且触发器外观正常,难以察觉,并可规避八种代表性先进防御机制。结果揭示了移动GUI代理中被忽视的后门向量,强调需加强对通知条件行为及内部代理表示的审查。
原文摘要 · Abstract (English)
The rapid adoption of mobile graphical user interface (GUI) agents, which autonomously control applications and operating systems (OS), exposes new system-level attack surfaces. Existing backdoors against web GUI agents and general GenAI models rely on environmental injection or deceptive pop-ups to mislead the agent operation. However, these techniques do not work on screenshots-based mobile GUI agents due to the challenges of restricted trigger design spaces, OS background interference, and conflicts in multiple trigger-action mappings. We propose AgentRAE, a novel backdoor attack capable of inducing Remote Action Execution in mobile GUI agents using visually natural triggers (e.g., benign app icons in notifications). To address the underfitting caused by natural triggers and achieve accurate multi-target action redirection, we design a novel two-stage pipeline that first enhances the agent's sensitivity to subtle iconographic differences via contrastive learning, and then associates each trigger with a specific mobile GUI agent action through a backdoor post-training. Our extensive evaluation reveals that the proposed backdoor preserves clean performance with an attack success rate of over 90% across ten mobile operations. Furthermore, it is hard to visibly detect the benign-looking triggers and circumvents eight representative state-of-the-art defenses. These results expose an overlooked backdoor vector in mobile GUI agents, underscoring the need for defenses that scrutinize notification-conditioned behaviors and internal agent representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。