首个面向隐私保护的GUI智能体评估基准，聚焦任务流程中的敏感信息泄露风险。
GUIGuard-Bench: Toward a General Evaluation for Privacy-Preserving GUI Agents
- 构建包含4080张截图的轨迹式隐私数据集，标注区域级隐私信息
- 现有模型能识别隐私存在但难以精确定位与判断风险必要性
- 适合研究隐私保护、GUI代理安全与人机交互的开发者和研究人员
随着GUI智能体越来越多依赖截图感知和操作数字环境，可能无意中暴露身份、账号、位置及行为痕迹等敏感信息。现有基准多关注任务完成度、定位能力或第三方攻击防御，但视觉隐私数据集仍主要局限于静态自然图像，难以捕捉GUI任务轨迹中的上下文依赖与任务相关性隐私风险。为此，我们提出首个面向轨迹式GUI工作流的隐私保护智能体评估基准——GUIGuard-Bench。该基准包含241条真实GUI智能体轨迹，覆盖安卓与PC环境，共4,080张截图，每张均在区域级别标注隐私边界框、语义类别、风险等级及信息是否任务必需。基于此标注，支持三项互补评估：隐私识别、受保护截图下的离线规划保真度，以及不同保护策略的效用影响。结果表明，当前模型虽可检测隐私存在，但在细粒度定位、类别识别、风险评估与任务必要性判断上表现不佳。同时发现，以Claude Sonnet 4.6为代表的闭源模型在应用隐私保护后，安卓环境中仍能保持较高规划语义一致性。研究凸显隐私识别是实际GUI智能体的关键瓶颈。
原文摘要 · Abstract (English)
As GUI agents increasingly rely on screenshots to perceive and operate digital environments, they may inadvertently expose sensitive information such as identities, accounts, locations, and behavioral traces. While existing benchmarks primarily focus on task completion, grounding, or defenses against third-party attacks, current visual privacy datasets remain largely restricted to static natural images, limiting their ability to capture the contextual dependence and task relevance of privacy risks in GUI task trajectories. To bridge this gap, we introduce \textbf{GUIGuard-Bench}, a first-step benchmark for studying privacy-preserving GUI agents in trajectory-based GUI workflows. GUIGuard-Bench contains 241 real GUI-agent trajectories with 4,080 screenshots across Android and PC environments. Each screenshot is annotated at the region level with privacy bounding boxes, semantic privacy categories, risk levels, and whether the private information is necessary for completing the task. Built on these annotations, GUIGuard-Bench supports three complementary evaluations: privacy recognition, offline planning fidelity under protected screenshots, and the utility impact of different protection strategies. Our results show that current models can often detect whether a screenshot contains private information, but they struggle with fine-grained localization, category recognition, risk assessment, and task-necessity judgment. We also find that closed-source models, exemplified by Claude Sonnet 4.6, can maintain largely consistent planner semantics in Android environments after privacy protection is applied. Our results highlight privacy recognition as a critical bottleneck for practical GUI agents. Project: https://futuresis.github.io/GUIGuard-page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。