arXiv:2606.15034cs.AI2026-06

评测计算机代理在真实任务中的安全风险,发现现有防护机制仍有漏洞。

OSGuard: A Benchmark for Safety in Computer-Use Agents

论文配图:OSGuard: A Benchmark for Safety in Computer-Use Agents
图 1 · 摘自论文原文
  • 构建双粒度安全评估基准,涵盖单步动作判断与全流程风险测试。
  • 在修改环境引入隐蔽危害后,仍能完成任务的案例中识别出不安全操作。
  • 适合研究智能代理安全、人机交互安全的开发者和研究人员参考。

计算机使用代理正越来越多地通过完成现实桌面与网页任务来评估。然而,仅关注任务成功可能忽略代理通过不安全捷径达成目标的情况。本文提出OSGuard,一个在无变化用户指令下评估计算机使用代理安全性的双粒度基准。该基准包含两部分:一是基于上下文的动作级评估,对提出的操作标记为允许、无关或不安全;二是风险增强执行套件,通过构造来自OSWorld的任务变体,在保持任务可完成的前提下引入潜在危害(如破坏性覆盖)。每个变体均配有增强评估器,在维持原始任务成功标准的同时,加入基于状态的安全不变量,从而区分安全完成与仅满足名义目标但不安全的完成。实验表明,当前多模态防护机制在孤立动作判断上表现良好,但在风险增强执行中暴露出局部监督与端到端可靠安全之间的差距。该双粒度设计使我们能更精准诊断模型是否既能识别不安全动作,又能作为防护机制提升整体任务安全性。

原文摘要 · Abstract (English)

Computer-use agents are increasingly evaluated by whether they complete realistic desktop and web tasks. However, task success alone can miss failures in which an agent reaches the nominal goal through an unsafe shortcut. We introduce OSGuard, a dual-granularity benchmark suite for evaluating safety in computer-use agents under benign, unchanged user instructions. OSGuard contains an action-level benchmark for local guardrail decisions and a risk-augmented execution suite for end-to-end evaluation. The action-level benchmark consists of contextualized proposed actions labeled as allowed, unrelated, or unsafe, each judged relative to the original instruction and current interface state. The execution suite contains manually constructed OSWorld-derived task variants in which the original task remains achievable, but the environment is modified to introduce latent hazards such as destructive overwrites, etc. Each variant is paired with augmented evaluators that retain the original task-success criterion while adding explicit state-based safety invariants, allowing us to distinguish safe completions from unsafe completions that satisfy the nominal task objective. Our experimental results on OSGuard show that current multimodal guardrails can perform well on isolated action judgments, while risk-augmented execution exposes remaining gaps between local oversight and reliable end-to-end safety. This dual-granularity design enables more precise diagnosis of whether models can both recognize unsafe proposed actions and improve full-task safety when deployed as guardrails.

智能代理安全评测风险检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。