发现网页智能体常无意泄露用户行为数据,比文本泄露多5倍。
SPILLage: Agentic Oversharing on the Web
- 提出双维度分类框架,区分内容与行为的显隐泄露。
- 实测1080次任务中行为泄露是文本泄露的5倍,且难以通过提示缓解。
- 清理无关信息可提升任务成功率17.9%,说明隐私保护影响效率。
LLM驱动的智能体在开放网络上自动化用户任务时,常访问邮件、日历等个人资源。与受控对话环境中的大模型不同,这些智能体在真实网站上“自由行动”,留下操作痕迹。本文提出自然代理过度共享(Natural Agentic Oversharing)——即通过网页操作痕迹无意泄露与任务无关的用户信息。我们构建了SPILLage框架,从通道(内容 vs. 行为)和直接性(显式 vs. 隐式)两个维度刻画泄露。在180个真实电商任务上进行1080次测试,覆盖两种代理框架和三种骨干LLM,结果表明:行为泄露(如点击、滚动、导航)是内容泄露的5倍,且即使在提示层面采取缓解措施,该问题依然存在甚至恶化。然而,在执行前移除任务无关信息,可使任务成功率最高提升17.9%,证明减少泄露有助于提升任务完成度。研究强调,网页智能体的隐私保护需超越传统输出视角,纳入其所有在线行为。
原文摘要 · Abstract (English)
LLM-powered agents are beginning to automate user's tasks across the open web, often with access to user resources such as emails and calendars. Unlike standard LLMs answering questions in a controlled ChatBot setting, web agents act "in the wild", interacting with third parties and leaving behind an action trace. Therefore, we ask the question: how do web agents handle user resources when accomplishing tasks on their behalf across live websites? In this paper, we formalize Natural Agentic Oversharing -- the unintentional disclosure of task-irrelevant user information through an agent trace of actions on the web. We introduce SPILLage, a framework that characterizes oversharing along two dimensions: channel (content vs. behavior) and directness (explicit vs. implicit). This taxonomy reveals a critical blind spot: while prior work focuses on text leakage, web agents also overshare behaviorally through clicks, scrolls, and navigation patterns that can be monitored. We benchmark 180 tasks on live e-commerce sites with ground-truth annotations separating task-relevant from task-irrelevant attributes. Across 1,080 runs spanning two agentic frameworks and three backbone LLMs, we demonstrate that oversharing is pervasive with behavioral oversharing dominates content oversharing by 5x. This effect persists -- and can even worsen -- under prompt-level mitigation. However, removing task-irrelevant information before execution improves task success by up to 17.9%, demonstrating that reducing oversharing improves task success. Our findings underscore that protecting privacy in web agents is a fundamental challenge, requiring a broader view of "output" that accounts for what agents do on the web, not just what they type. Our datasets and code are available at https://github.com/jrohsc/SPILLage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。