用户隐私保护下实现高质量个性化生成，不暴露个人资料。
Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations
- 客户端用私有资料修改服务器生成的草稿，反复迭代优化输出。
- 相比非个性化模型，性能提升7.4%至9%，接近全暴露隐私时的90%以上效果。
- 适合注重隐私的个性化应用，如医疗、金融对话系统。
个性化对齐大语言模型输出与用户偏好至关重要。现有方法在暴露私有数据与依赖弱本地模型间权衡。我们提出 $P^3$ 框架：服务器端仅根据用户查询生成 $k$ 个草稿词元，客户端利用私有资料检索并修改草稿以更好反映用户偏好，重复此过程直至生成结束符。在包含三个个性化问答数据集的 LaMP-QA 基准测试中，$P^3$ 持续优于非个性化服务器端和个性化客户端基线，平均提升7.4%至9%。重要的是，$P^3$ 恢复了全暴露情景下90.3%至95.7%的性能。隐私分析表明，$P^3$ 保护隐私，相比无上下文查询仅增加1.5%至3.5%的泄露风险。框架高效，客户端仅生成总词元的9.2%。结果证明 $P^3$ 在保障隐私的同时提供实用有效的个性化生成方案。
原文摘要 · Abstract (English)
Personalization is crucial for aligning Large Language Model (LLM) outputs with individual user preferences and background knowledge. State-of-the-art solutions are based on retrieval augmentation, where relevant context from a user profile is retrieved for LLM consumption. These methods deal with a trade-off between exposing retrieved private data to cloud providers and relying on less capable local models. We introduce $P^3$, an interactive framework for high-quality personalization without revealing private profiles to server-side LLMs. In $P^3$, a large server-side model generates a sequence of $k$ draft tokens based solely on the user query, while a small client-side model, with retrieval access to the user's private profile, evaluates and modifies these drafts to better reflect user preferences. This process repeats until an end token is generated. Experiments on LaMP-QA, a recent benchmark consisting of three personalized question answering datasets, show that $P^3$ consistently outperforms both non-personalized server-side and personalized client-side baselines, achieving statistically significant improvements of $7.4%$ to $9%$ on average. Importantly, $P^3$ recovers $90.3%$ to $95.7%$ of the utility of a ``leaky'' upper-bound scenario in which the full profile is exposed to the large server-side model. Privacy analyses, including linkability and attribute inference attacks, indicate that $P^3$ preserves the privacy of a non-personalized server-side model, introducing only marginal additional leakage ($1.5%$--$3.5%$) compared to submitting a query without any personal context. Additionally, the framework is efficient for edge deployment, with the client-side model generating only $9.2%$ of the total tokens. These results demonstrate that $P^3$ provides a practical, effective solution for personalized generation with improved privacy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。