提出隐私保护的上下文学习框架，通过邻近搜索提升效率与安全性。
Differentially Private In-Context Learning with Nearest Neighbor Search
- 基于邻近搜索从数据中检索相关样本，结合隐私预算追踪机制。
- 在文本分类和文档问答任务上显著优于现有基线方法。
- 适合关注大模型隐私保护的开发者与研究者使用。
差分隐私的上下文学习（DP-ICL）因上下文学习固有的隐私风险而成为研究热点。然而，现有方法忽视了现代大语言模型流水线中的关键环节：用于检索相关上下文数据的相似性搜索。本文提出一种隐私感知的DP框架，将上下文数据的最近邻检索与隐私过滤相结合，以确保符合中心差分隐私预算。该方法通过跟踪所选样本的累积隐私成本，实现更优的隐私-效用权衡。在文本分类与文档问答任务上的实验表明，本方法在所有评估基准上均显著优于现有基线，展现出明显优势。
原文摘要 · Abstract (English)
Differentially private in-context learning (DP-ICL) has recently become an active research topic due to the inherent privacy risks of in-context learning. However, existing approaches overlook a critical component of modern large language model (LLM) pipelines: the similarity search used to retrieve relevant context data. In this work, we introduce a DP framework for in-context learning that integrates nearest neighbor search of relevant examples in a privacy-aware manner. Our method outperforms existing baselines by a substantial margin across all evaluated benchmarks, achieving more favorable privacy-utility trade-offs. To achieve this, we employ nearest neighbor retrieval from a database of context data, combined with a privacy filter that tracks the cumulative privacy cost of selected samples to ensure adherence to a central differential privacy budget. Experimental results on text classification and document question answering show a clear advantage of the proposed method over existing baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。