测试智能代理在跨应用操作中的隐私泄露风险，发现多数存在严重信息外泄。
Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?
- 构建可量化评估的AgentCIBench测试框架，识别三类隐私泄露场景。
- 15个前沿代理中11个在超半数场景泄露信息，平均泄露率达67.9%。
- 适用于关注AI代理安全性的研究者与开发者，推动上线前隐私检测。
计算机使用代理（CUAs）现可在邮件、日历、待办事项等个人应用中代表用户操作。这种跨应用访问虽便利，却带来被忽视的隐私风险：当代理在某一上下文中工作时，可能获取另一上下文中的不恰当信息。为此，我们提出AgentCIBench评估框架，将该风险转化为可执行、可确定评分的场景。聚焦三类常见失败模式：视觉共位（代理调取界面中邻近任务目标的禁止项）、任务模糊过度共享（对不明确指令返回密集个人状态）、收件人错配（向不合适的接收者发送内容）。评估15个前沿代理发现，11个在超过50%的场景中泄露信息，平均泄露率为67.9%，且在端到端环境任务中失败仍持续存在。我们发布AgentCIBench，以促进更安全的计算机使用代理开发，并将上下文披露测试定位为部署前的安全检查。
原文摘要 · Abstract (English)
Computer-use agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-application access is useful, but it also creates a privacy risk that has been largely overlooked: when an agent works in one context, it can pull in information from another that is inappropriate in that context. Hence, we introduce AgentCIBench, an evaluation harness that turns this risk into executable, deterministically scored scenarios. We target three common failure modes in CUAs: visual co-location, where the agent pulls in prohibited items that sit next to the task target in the UI; task-ambiguity overshare, where the agent dumps dense personal state in response to an under-specified prompt; and recipient misalignment, where the agent sends content to an addressee for whom it is inappropriate. We evaluate 15 frontier agents and find a surprisingly high failure rate: 11 of 15 leak on more than 50% of scenarios, with an average leakage of 67.9%, and the same failures persist when agents act end-to-end in the environment to complete the task. We release AgentCIBench to encourage the development of safer computer-use agents and position contextual disclosure testing as a pre-deployment safety check.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。