研究生成式AI代理泄露企业数据隐私的机制并给出最优防护方案
Differential Privacy in Generative AI Agents: Analysis and Optimal Tradeoffs
- 构建基于差分隐私的生成机制分析框架
- 发现温度与消息长度影响隐私泄露程度，可量化关联
- 提供隐私与效果权衡下的温度优化选择方法
大型语言模型和AI代理正被广泛集成到企业系统中，用于访问内部数据库并生成上下文相关响应。尽管提升了生产力与决策支持能力，但模型输出可能无意间泄露敏感信息。现有研究多关注用户提示的隐私保护，较少从企业数据角度考虑隐私风险。本文提出一种概率框架，基于差分隐私分析AI代理中的隐私泄露问题。将响应生成建模为从提示与数据集映射到词元序列分布的随机机制，在此框架下引入词元级与消息级差分隐私，并推导出隐私边界，揭示隐私泄露与生成参数（如温度、消息长度）之间的关系。进一步构建隐私-效用设计问题，实现最优温度选择。
原文摘要 · Abstract (English)
Large language models (LLMs) and AI agents are increasingly integrated into enterprise systems to access internal databases and generate context-aware responses. While such integration improves productivity and decision support, the model outputs may inadvertently reveal sensitive information. Although many prior efforts focus on protecting the privacy of user prompts, relatively few studies consider privacy risks from the enterprise data perspective. Hence, this paper develops a probabilistic framework for analyzing privacy leakage in AI agents based on differential privacy. We model response generation as a stochastic mechanism that maps prompts and datasets to distributions over token sequences. Within this framework, we introduce token-level and message-level differential privacy and derive privacy bounds that relate privacy leakage to generation parameters such as temperature and message length. We further formulate a privacy-utility design problem that characterizes optimal temperature selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。