arXiv:2602.01725cs.CLcs.AI2026-02被引 6

用世界模型预判长期风险,让电脑代理提前避开危险操作

SafePred: A Predictive Guardrail for Computer-Using Agents via World Models

  • 基于世界模型预测未来风险,建立风险与决策的闭环
  • 在97.6%以上场景下避免高危行为,任务效率提升21.4%
  • 适合需要长期安全的自动化系统,如运维、金融代理

随着计算机使用代理(CUAs)在复杂真实环境中的广泛应用,长期风险常导致严重且不可逆的后果。现有防护机制多为被动响应,仅在当前观察空间内限制行为,无法预防延迟出现的长期风险——看似合理的操作可能引发未来高风险结果(如删除日志导致审计无法追溯),而被动防护难以识别此类问题。为此,本文提出预测性防护框架SafePred,核心思想是将预测的未来风险与当前决策对齐。SafePred具备两项关键能力:(1) 短期与长期风险预测:基于安全策略,利用世界模型生成短期与长期风险的语义表征,识别并剔除导致高风险状态的动作;(2) 决策优化:通过步骤级干预和任务级重规划,将预测风险转化为可执行的安全决策指引。大量实验表明,SafePred显著降低高危行为,在超过97.6%的场景中实现安全防护,同时相比被动基线,任务效用最高提升21.4%。

原文摘要 · Abstract (English)

With the widespread deployment of Computer-using Agents (CUAs) in complex real-world environments, prevalent long-term risks often lead to severe and irreversible consequences. Most existing guardrails for CUAs adopt a reactive approach, constraining agent behavior only within the current observation space. While these guardrails can prevent immediate short-term risks (e.g., clicking on a phishing link), they cannot proactively avoid long-term risks: seemingly reasonable actions can lead to high-risk consequences that emerge with a delay (e.g., cleaning logs leads to future audits being untraceable), which reactive guardrails cannot identify within the current observation space. To address these limitations, we propose a predictive guardrail approach, with the core idea of aligning predicted future risks with current decisions. Based on this approach, we present SafePred, a predictive guardrail framework for CUAs that establishes a risk-to-decision loop to ensure safe agent behavior. SafePred supports two key abilities: (1) Short- and long-term risk prediction: by using safety policies as the basis for risk prediction, SafePred leverages the prediction capability of the world model to generate semantic representations of both short-term and long-term risks, thereby identifying and pruning actions that lead to high-risk states; (2) Decision optimization: translating predicted risks into actionable safe decision guidances through step-level interventions and task-level re-planning. Extensive experiments show that SafePred significantly reduces high-risk behaviors, achieving over 97.6% safety performance and improving task utility by up to 21.4% compared with reactive baselines.

安全代理世界模型风险预测自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。