GUI智能体在本地动态决定屏幕隐私处理方式，避免敏感信息外泄。
MaskClaw: Edge-Side Personalized Privacy Arbitration for GUI Agents with Behavior-Driven Skill Evolution
- 在设备端基于任务和用户角色判断是否屏蔽或询问隐私内容
- 通过五种场景演化出可复用的隐私处理技能，准确率超基准方法23%
- 适合需要高隐私保护的办公、医疗等敏感场景使用
GUI智能体依赖截图理解意图并跨应用操作，但截图常含私密消息、医疗记录、支付凭证及工作流程。隐私决策需考虑任务、接收方、应用状态与用户角色，而静态PII检测器难以识别边界，云端大模型推理又可能在决策前上传原始屏幕。我们提出MaskClaw，一种边缘侧隐私仲裁机制：提取本地视觉证据，检索用户与任务相关的策略记忆，在截图离开受控环境前决定允许、遮蔽或询问。在五个设计的技能演化场景中，修正、取消与编辑被转化为可复用的隐私技能，并由沙箱门控验证。我们构建了P-GUI-Evo基准，包含真实UI模式、重构的HTML屏幕与清洗标签。实验表明，仅靠模式匹配、云端推理或路由策略易导致过度确认、过度遮蔽或暴露原始屏幕。项目代码已公开于https://github.com/Theodora-Y/MaskClaw。
原文摘要 · Abstract (English)
GUI agents rely on screenshots to infer intent and operate across applications, but these screenshots often contain private messages, medical records, payment credentials, and workplace-specific workflows. Privacy decisions in this setting depend on task, recipient, application state, and user role, yet static PII detectors miss these boundaries and cloud-side VLM reasoning can upload the raw screen before deciding what should be protected. We present MaskClaw, an edge-side privacy arbitrator for GUI agents. MaskClaw extracts local visual evidence, retrieves user- and task-specific policy memory, and decides Allow, Mask, or Ask before raw screenshots leave a trusted user- or organization-controlled environment. In five designed skill-evolution scenarios, it turns corrections, cancellations, and edits into reusable privacy skills checked by a sandbox gate. We introduce P-GUI-Evo, a benchmark built from real UI patterns, reconstructed HTML screens, and sanitized labels. Experiments show that pattern matching, cloud reasoning, and routing alone tend to over-confirm, over-mask, or expose raw screenshots under the same protocol. The artifact is available at https://github.com/Theodora-Y/MaskClaw.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。