提出token级隐私保护机制，兼顾语义与上下文信息，提升语言模型服务的隐私安全
Token-Level Privacy in Large Language Models
- 融合语义与上下文信息，实现dchi差分隐私下的强隐私保障
- 在多个数据集上达成优于或相当现有方法的隐私-效用平衡
- 适用于高风险场景下需保护用户输入隐私的语言模型服务
将语言模型作为远程服务使用时，需将私密信息传至外部提供方，引发重大隐私担忧。该过程不仅可能使敏感数据暴露于不可信服务方，还易被窃听者截获。现有NLP隐私保护方法多依赖语义相似性，忽视了上下文信息的作用。本文提出dchi-stencil，一种新型的token级隐私保护机制，在dchi差分隐私框架下同时整合语义与上下文信息，实现2epsilon-dchi-隐私保障。通过融合语义与上下文细节，dchi-stencil在隐私与效用之间取得稳健平衡。我们在先进语言模型和多样数据集上评估该方法，结果表明其在隐私-效用权衡方面达到或超过现有方法。本工作展示了dchi-stencil在现代高风险应用中设定隐私保护新标准的潜力。
原文摘要 · Abstract (English)
The use of language models as remote services requires transmitting private information to external providers, raising significant privacy concerns. This process not only risks exposing sensitive data to untrusted service providers but also leaves it vulnerable to interception by eavesdroppers. Existing privacy-preserving methods for natural language processing (NLP) interactions primarily rely on semantic similarity, overlooking the role of contextual information. In this work, we introduce dchi-stencil, a novel token-level privacy-preserving mechanism that integrates contextual and semantic information while ensuring strong privacy guarantees under the dchi differential privacy framework, achieving 2epsilon-dchi-privacy. By incorporating both semantic and contextual nuances, dchi-stencil achieves a robust balance between privacy and utility. We evaluate dchi-stencil using state-of-the-art language models and diverse datasets, achieving comparable and even better trade-off between utility and privacy compared to existing methods. This work highlights the potential of dchi-stencil to set a new standard for privacy-preserving NLP in modern, high-risk applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。