RAG在医疗问答中易受通用投毒攻击,可被恶意文档误导。
On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
- 通过构造含敏感信息的投毒文档,攻击者可诱导任意查询精准命中恶意内容。
- 在225种组合下,攻击成功率接近100%,因查询与投毒文档嵌入相似性具规律性。
- 提出基于检测的防御方法,在多领域实验中检测率近乎全胜,适合高风险应用者。
检索增强生成(RAG)在医疗、金融、法律等知识密集型领域被证明能提升大语言模型性能。其工作方式是:针对用户查询,从语料库中检索相关文档并融入生成过程。本文研究RAG的对抗鲁棒性,聚焦检索系统脆弱性。在225种不同语料、检索器、查询和目标信息组合下,我们发现医疗问答场景中检索系统易受通用投毒攻击:攻击者生成包含广泛目标信息(如个人身份信息)的投毒文档,一旦插入语料库,只要使用指定查询,即可被精确检索到。分析表明,查询嵌入与投毒文档嵌入间的偏差具有规律性,从而维持高相似度,确保精准命中。基于此,我们提出一种新型检测防御机制。在跨多个问答领域的大量实验中,该方法在几乎所有情况下均实现优异检测率。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has been empirically shown to enhance the performance of large language models (LLMs) in knowledge-intensive domains such as healthcare, finance, and legal contexts. Given a query, RAG retrieves relevant documents from a corpus and integrates them into the LLMs' generation process. In this study, we investigate the adversarial robustness of RAG, focusing specifically on examining the retrieval system. First, across 225 different setup combinations of corpus, retriever, query, and targeted information, we show that retrieval systems are vulnerable to universal poisoning attacks in medical Q\&A. In such attacks, adversaries generate poisoned documents containing a broad spectrum of targeted information, such as personally identifiable information. When these poisoned documents are inserted into a corpus, they can be accurately retrieved by any users, as long as attacker-specified queries are used. To understand this vulnerability, we discovered that the deviation from the query's embedding to that of the poisoned document tends to follow a pattern in which the high similarity between the poisoned document and the query is retained, thereby enabling precise retrieval. Based on these findings, we develop a new detection-based defense to ensure the safe use of RAG. Through extensive experiments spanning various Q\&A domains, we observed that our proposed method consistently achieves excellent detection rates in nearly all cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。