用差分隐私增强大模型推理隐私，提升数据保护与生成质量的平衡。
Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy
- 基于上下文感知的提示扰动，结合差分隐私保护用户数据。
- 在多个数据集上实现更优的隐私-效用权衡，优于现有先进方法。
- 适合关注大模型隐私安全的研究者与开发者使用。
大型语言模型（LLMs）因其出色的文本理解与生成能力而广受欢迎。然而，在ChatGPT等推理服务中的广泛应用引发了敏感用户数据泄露的担忧。现有方案多依赖隐私增强技术，但面临效率、隐私与实用性之间的权衡。为此，我们提出Cape——一种基于差分隐私的上下文感知提示扰动机制，可在保持高效推理的同时改善隐私-效用平衡。具体而言，我们设计了一种混合效用函数以更准确捕捉词元相似性，并提出分桶采样机制以应对大规模采样空间带来的长尾现象。在多个数据集上的大量实验及消融研究显示，Cape相较于现有最先进方法实现了更优的隐私-效用权衡。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have gained significant popularity due to their remarkable capabilities in text understanding and generation. However, despite their widespread deployment in inference services such as ChatGPT, concerns about the potential leakage of sensitive user data have arisen. Existing solutions primarily rely on privacy-enhancing technologies to mitigate such risks, facing the trade-off among efficiency, privacy, and utility. To narrow this gap, we propose Cape, a context-aware prompt perturbation mechanism based on differential privacy, to enable efficient inference with an improved privacy-utility trade-off. Concretely, we introduce a hybrid utility function that better captures the token similarity. Additionally, we propose a bucketized sampling mechanism to handle large sampling space, which might lead to long-tail phenomenons. Extensive experiments across multiple datasets, along with ablation studies, demonstrate that Cape achieves a better privacy-utility trade-off compared to prior state-of-the-art works.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。