为AI助手设计安全环境,确保用户隐私数据不被泄露。
An AI Agent Execution Environment to Safeguard User Data
- 通过动态提示收集用户数据共享权限并强制执行。
- 可阻止所有数据泄露攻击,且不影响助手功能。
- 无需信任模型或提示,适合隐私敏感场景使用。
AI代理有望成为通用个人助手,但需访问用户私密数据(如个人和财务信息),存在严重安全与隐私风险。攻击者可能通过提示注入等手段窃取数据。此外,用户需信任不可靠或已被攻破的模型提供方。本文提出GAAP(Guaranteed Accounting for Agent Privacy)——一种保障代理隐私的执行环境,通过动态定向用户提示收集数据共享权限,并确保代理在执行中对用户数据的披露(包括向模型及其提供方)严格遵守这些规范。关键在于,GAAP以确定性方式提供保障,无需信任代理、模型或用户提示,也不要求其无攻击漏洞。它通过增强信息流控制,引入新型持久化数据存储与标注,实现跨任务及单任务内对私密信息流动的追踪。评估表明,GAAP能完全阻断所有数据泄露攻击,优于现有最先进系统,且对代理实用性影响极小。
原文摘要 · Abstract (English)
AI agents promise to serve as general-purpose personal assistants for their users, which requires them to have access to private user data (e.g., personal and financial information). This poses a serious risk to security and privacy. Adversaries may attack the AI model (e.g., via prompt injection) to exfiltrate user data. Furthermore, sharing private data with an AI agent requires users to trust a potentially unscrupulous or compromised AI model provider with their private data. This paper presents GAAP (Guaranteed Accounting for Agent Privacy), an execution environment for AI agents that guarantees confidentiality for private user data. Through dynamic and directed user prompts, GAAP collects permission specifications from users describing how their private data may be shared, and GAAP enforces that the agent's disclosures of private user data, including disclosures to the AI model and its provider, comply with these specifications. Crucially, GAAP provides this guarantee deterministically, without trusting the agent with private user data, and without requiring any AI model or the user prompt to be free of attacks. GAAP enforces the user's permission specification by tracking how the AI agent accesses and uses private user data. It augments Information Flow Control with novel persistent data stores and annotations that enable it to track the flow of private information both across execution steps within a single task, and also over multiple tasks separated in time. Our evaluation confirms that GAAP blocks all data disclosure attacks, including those that make other state-of-the-art systems disclose private user data to untrusted parties, without a significant impact on agent utility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。