通过预判操作后果，防止手机智能助手误操作导致不可逆损失。
SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction
- 构建统一世界模型，提前预测操作后果并评估风险
- 在Qwen3-VL-8B上将安全效率提升至0.596，风险成本降至0.130
- 适合需要高安全性的手机自动化任务场景
移动端图形用户界面（GUI）智能体在自动化复杂任务方面表现卓越，但单一错误操作可能引发不可逆后果。现有安全机制多为被动响应，缺乏执行前的风险评估能力。本文提出SeerGuard，一种基于世界模型预测的主动安全框架，通过指令级筛查与动作级风险评估，在执行前识别潜在风险。为此，我们采用多任务学习构建安全增强型世界模型（SAWM），融合语义下一状态预测与安全风险评估。大量实验表明，SeerGuard能有效泛化于多种移动端智能体。在Qwen3-VL-8B-Instruct上，当ω=0.8时，安全-效用得分从0.191提升至0.596；当α=0.8时，风险成本从0.347降至0.130。对SAWM的进一步分析验证了指令级筛查的有效性，以及动作风险评估与状态预测能力。
原文摘要 · Abstract (English)
Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safety risks where a single erroneous action can lead to irreversible consequences. Existing safety mechanisms are primarily reactive, lacking the ability to assess risks before execution. In this paper, we introduce SeerGuard, a consequence-aware safety framework designed to mitigate these risks through pre-execution instruction-level screening and action-level risk assessment. Specifically, the action-level assessment analyzes agent-proposed actions within current GUI states, anticipating likely outcomes to identify risks before they are executed. To enable these capabilities, we construct a unified safety-augmented world model (SAWM) via multi-task learning, integrating semantic next-state prediction with safety risk assessment. Extensive experiments demonstrate that SeerGuard generalizes effectively across diverse mobile GUI agents. On Qwen3-VL-8B-Instruct, it increases the safety-utility score from $0.191$ to $0.596$ at $ω=0.8$ and reduces the risk-cost score from $0.347$ to $0.130$ at $α=0.8$. Further analyses on our SAWM validate the effectiveness of the instruction-level screening, alongside the capability of action risk assessment and next-state prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。