让大模型在检索时自动抹除私密信息,保护隐私同时保留可用知识。
Learning to Erase Private Knowledge from Multi-Documents for Retrieval-Augmented Large Language Models
- 构建全局知识图谱识别跨文档敏感信息,防脱敏攻击。
- 用Flan-T5重写文档,精准删除指定私密内容,保留公共知识。
- 适合需隐私保护的行业应用,如医疗、金融领域RAG系统。
检索增强生成(RAG)是将大语言模型应用于专有领域的有效方法,但检索到的文档可能包含敏感信息,导致生成结果泄露隐私。因此,如何从检索文档中有效擦除私密知识,成为RAG的关键挑战。与传统文本匿名化不同,RAG需考虑:(1) 多文档推理可能引发去匿名化攻击;(2) 私密知识因场景而异,需支持用户自定义删除内容;(3) 保留足够公开知识以支撑生成任务。本文提出RAG的隐私擦除任务,并设计Eraser4RAG,一种可有效移除用户指定私密知识的同时保留必要公共知识的擦除器。首先构建全局知识图谱,识别跨文档潜在知识,防范去匿名化攻击;随后随机划分图谱为私有与公共子图,微调Flan-T5模型重写文档,剔除私有三元组;最后使用PPO算法优化重写模型,最小化私有三元组保留,最大化公共三元组保留。在四个QA数据集上的实验表明,Eraser4RAG在擦除性能上优于GPT-4o。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is a promising technique for applying LLMs to proprietary domains. However, retrieved documents may contain sensitive knowledge, posing risks of privacy leakage in generative results. Thus, effectively erasing private information from retrieved documents is a key challenge for RAG. Unlike traditional text anonymization, RAG should consider: (1) the inherent multi-document reasoning may face de-anonymization attacks; (2) private knowledge varies by scenarios, so users should be allowed to customize which information to erase; (3) preserving sufficient publicly available knowledge for generation tasks. This paper introduces the privacy erasure task for RAG and proposes Eraser4RAG, a private knowledge eraser which effectively removes user-defined private knowledge from documents while preserving sufficient public knowledge for generation. Specifically, we first construct a global knowledge graph to identify potential knowledge across documents, aiming to defend against de-anonymization attacks. Then we randomly split it into private and public sub-graphs, and fine-tune Flan-T5 to rewrite the retrieved documents excluding private triples. Finally, PPO algorithm optimizes the rewriting model to minimize private triples and maximize public triples retention. Experiments on four QA datasets demonstrate that Eraser4RAG achieves superior erase performance than GPT-4o.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。