通过分组文本重写实现提示隐私保护,兼顾隐私与模型性能。
DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting
- 采用三阶段框架,结合局部差分隐私与上下文学习,同时处理词级和文档级信息。
- 在CommonSense QA和DocVQA上优于现有方法,实现更优的隐私-效用权衡。
- 可作为插件集成到现有重写技术中,适合关注提示隐私的开发者使用。
提示隐私对在线大语言模型至关重要,因提示常包含敏感信息。尽管大语言模型可通过文本重写提升隐私性,但现有方法多集中于文档级重写,忽略了文本的多层次表征。这一局限使大语言模型难以泛化,忽视其上下文学习能力,阻碍实际应用。为此,我们提出DP-GTR,一种基于局部差分隐私(DP)与组合定理的三阶段分组文本重写框架。DP-GTR首次融合文档级与词级信息,利用上下文学习,在个体数据点层面有效衔接局部与全局差分隐私机制,同时提升隐私与效用。在CommonSense QA和DocVQA上的实验表明,该方法优于现有方案,实现了更优的隐私-效用平衡。此外,框架兼容现有重写技术,可作为插件增强隐私保护。代码已公开于github.com/ResponsibleAILab/DP-GTR。
原文摘要 · Abstract (English)
Prompt privacy is crucial, especially when using online large language models (LLMs), due to the sensitive information often contained within prompts. While LLMs can enhance prompt privacy through text rewriting, existing methods primarily focus on document-level rewriting, neglecting the rich, multi-granular representations of text. This limitation restricts LLM utilization to specific tasks, overlooking their generalization and in-context learning capabilities, thus hindering practical application. To address this gap, we introduce DP-GTR, a novel three-stage framework that leverages local differential privacy (DP) and the composition theorem via group text rewriting. DP-GTR is the first framework to integrate both document-level and word-level information while exploiting in-context learning to simultaneously improve privacy and utility, effectively bridging local and global DP mechanisms at the individual data point level. Experiments on CommonSense QA and DocVQA demonstrate that DP-GTR outperforms existing approaches, achieving a superior privacy-utility trade-off. Furthermore, our framework is compatible with existing rewriting techniques, serving as a plug-in to enhance privacy protection. Our code is publicly available at github.com/ResponsibleAILab/DP-GTR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。