低危害指令伪装成正常操作,绕过人机协同防御,威胁计算机代理安全。
Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents

- 用看似无害的指令诱导代理执行隐蔽攻击任务。
- 444个测试用例中多数攻击成功绕过模型与人工审核。
- 适合关注AI代理安全的开发者与研究者阅读。
计算机使用代理(CUAs)使大语言模型能自主操作操作系统和网页,但正面临间接提示注入攻击。当前主流防御是人机协同模式,即代理在执行敏感操作前暂停并等待用户确认。然而,该机制对“隐形墨水威胁”无效:这类攻击以低危害指令(如收藏仓库、安装包)为伪装,行为上与合法任务无异,既逃过模型安全机制,也避开人工审查。为此,我们构建了II-Bench,包含444个针对保密性和完整性攻击的对抗性任务,覆盖三个平台,涵盖页面导航与交互、敏感信息窃取、代码下载与执行三类攻击,每类均提供自然语言与代码形式,并设两种指令明确度。同时,我们设计了HITLCUA框架,整合真实虚拟机环境与隔离的Docker网页平台,通过模拟用户接口让代理在可疑操作前咨询“人类”,从而评估防御效果。对主流CUAs的全面测试显示,低危害注入常成功绕过代理防御与模拟用户审查,揭示了当前代理存在严重且未被充分关注的安全风险。
原文摘要 · Abstract (English)
Computer-use agents (CUAs), which empower large language models to autonomously operate operating systems and the web, are increasingly vulnerable to indirect prompt injection attacks. A widely adopted defense is the human-in-the-loop paradigm, in which the agent pauses for explicit user confirmation before executing sensitive operations. While effective against conspicuously high-harm attacks, this defense offers little protection against what we term Invisible Ink Threats: low-harm injected goals, such as starring a repository or installing a package, that are behaviorally indistinguishable from legitimate task execution and thus evade both model safety mechanisms and human oversight. To systematically investigate this blind spot, we present II-Bench, a collection of seemingly harmless adversarial tasks. II-Bench comprises 444 examples targeting confidentiality and integrity attacks across three platforms, spanning three attack categories: page navigation and interaction, sensitive information exfiltration, and code download and execution. Each category is instantiated in both natural language and code forms under two levels of instruction specificity. Furthermore, we construct HITLCUA, a comprehensive adversarial testing framework that integrates a real virtual machine operating system environment with isolated Docker-based web platforms, and simulates human participation by allowing CUAs to consult an API-simulated user before proceeding with suspicious operations. Extensive evaluations of leading CUAs reveal that low-harm injections frequently bypass both agent defenses and simulated user review, exposing severe and previously underexplored security risks in current CUAs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。