用稀疏注意力防御RAG被恶意文档误导
Addressing Corpus Knowledge Poisoning Attacks on RAG Using Sparse Attention
- 设计块稀疏注意力，禁止检索文档间交互
- 在多种攻击下显著提升问答准确率
- 可无缝集成现有防御方法，效果更优
检索增强生成（RAG）能有效保持大模型输出更新并减少幻觉，但近期研究发现其易受语料库知识投毒攻击：攻击者向语料库注入误导性文档，诱导大模型生成错误回答。我们指出，标准因果注意力机制导致有害的跨文档信息交互，尤其在攻击场景下。为此提出新防御方法：稀疏文档注意力RAG（SDAG），采用块稀疏注意力机制，禁止检索文档间的交叉注意力。该方法仅需在推理时微调注意力掩码。我们在多种攻击策略下对基于大模型的问答任务进行了实证评估，结果表明SDAG显著优于标准因果注意力机制。进一步证明，将SDAG与当前最先进的RAG防御方法结合后，性能在统计上显著超越现有最优方案。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) is a highly effective paradigm for keeping LLM-based responses up-to-date and reducing the likelihood of hallucinations. Yet, RAG was recently shown to be quite vulnerable to corpus knowledge poisoning: an attacker injects misleading documents to the corpus to steer an LLM's output to an undesired response. We argue that the standard causal attention mechanism in LLMs enables harmful cross-document interactions, specifically in cases of attacks. Accordingly, we introduce a novel defense approach for RAG: Sparse Document Attention RAG (SDAG). This is a block-sparse attention mechanism that disallows cross-attention between retrieved documents. SDAG requires a minimal inference-time change to the attention mask. We present an empirical evaluation of LLM-based question answering (QA) with a variety of attack strategies on RAG. We show that our SDAG method substantially outperforms the standard causal attention mechanism. We further demonstrate the clear merits of integrating SDAG with state-of-the-art RAG defense methods. Specifically, the integration results in performance that is statistically significantly better than the state-of-the-art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。