arXiv:2601.09923cs.AI2026-01被引 15

提出安全框架让电脑操作智能体在不被操控的前提下完成任务。

CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents

  • 用一次性规划确保执行流程不受恶意指令干扰。
  • 在OSWorld上保持前沿模型57%性能,开源模型提升19%。
  • 适合关注智能体安全与实用平衡的研究者和开发者。

AI智能体易受提示注入攻击,攻击者可通过恶意内容劫持其行为。现有防御中,架构隔离通过严格分离可信的任务规划与不可信的环境观测提供最强保障。然而,对于依赖持续观察界面状态来决策动作的电脑操作智能体(CUAs),这一设计存在根本矛盾。本文发现,尽管用户界面流程动态变化,但其结构具有可预测性。通过单次规划——即可信规划器提前生成覆盖所有预期运行时状态的完整分支计划——可实现对控制流的完整性保护,抵御任意指令注入。我们提出NOVA(导航、验证、行动)系统,使该方案在组合爆炸的界面状态空间中可行,计划可调用感知模型解析运行时值(如界面坐标)。在OSWorld上的评估显示,该设计在保留前沿模型57%性能的同时,使小型开源模型性能提升最高达19%,证明了严格安全与实用性能可共存。尽管前置规划能防止指令注入,我们进一步揭示需额外措施应对'分支引导'攻击——攻击者诱导感知模型将执行路径引向攻击者偏好的分支,例如跳转至恶意网站。

原文摘要 · Abstract (English)

AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guarantees by strictly separating trusted task planning from untrusted environment observations. However, applying this design to Computer Use Agents (CUAs), which automate tasks by viewing screens and executing actions, presents a fundamental challenge. Current agents require continuous observation of UI state to determine each action, which conflicts with the isolation required for security. We resolve this tension by demonstrating that UI workflows, while dynamic, are structurally predictable. Single-shot planning, where a trusted planner emits upfront a complete branching plan covering all anticipated runtime states, provides control flow integrity guarantees against arbitrary instruction injections. We introduce NOVA (Navigating via Observation, Verification, and Action) to make this viable in the combinatorially large UI state space, where the plan can invoke a perception model to resolve runtime values such as UI coordinates. We evaluate our design on OSWorld, and retain up to 57% of the performance of frontier models while improving performance for smaller open-source models by up to 19%, demonstrating that rigorous security and utility can coexist in CUAs. Although upfront planning prevents instruction injections, we show that additional measures are needed to defend against \textbf{Branch Steering} attacks, where adversaries deceive the perception model into routing execution down attacker-preferred branches of the plan, such as redirecting the agent to a malicious website.

智能体安全计算机操作提示注入系统级防护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。