用思维链模板伪造知识,欺骗具备深度推理的RAG系统。
Chain-of-Thought Poisoning Attacks against R1-based Retrieval-Augmented Generation Systems
- 用模型自身思维模式生成误导性文档注入知识库。
- 在MS MARCO数据集上成功诱导RAG系统输出错误答案。
- 适合研究RAG安全性和对抗攻击的开发者参考。
检索增强生成(RAG)系统能有效缓解大语言模型的幻觉问题,但也存在固有漏洞。在大规模部署前识别这些弱点至关重要,有助于构建更安全可靠的RAG系统。现有攻击方法多通过知识库污染探测漏洞,可有效欺骗传统RAG模型。然而,随着现代大模型深度推理能力的提升,仅注入错误知识的方法已不足以为继。本文受大模型深度思考能力启发,从基于R1的RAG系统中提取推理过程模板,将其与错误知识结合生成对抗性文档,并注入知识库以攻击具备深度推理能力的RAG系统。核心思想是:对抗文档若模拟了模型训练信号中的思维链模式,可能被模型误认为真实的历史推理过程,从而提高被引用概率。在MS MARCO段落排序数据集上的实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems can effectively mitigate the hallucination problem of large language models (LLMs),but they also possess inherent vulnerabilities. Identifying these weaknesses before the large-scale real-world deployment of RAG systems is of great importance, as it lays the foundation for building more secure and robust RAG systems in the future. Existing adversarial attack methods typically exploit knowledge base poisoning to probe the vulnerabilities of RAG systems, which can effectively deceive standard RAG models. However, with the rapid advancement of deep reasoning capabilities in modern LLMs, previous approaches that merely inject incorrect knowledge are inadequate when attacking RAG systems equipped with deep reasoning abilities. Inspired by the deep thinking capabilities of LLMs, this paper extracts reasoning process templates from R1-based RAG systems, uses these templates to wrap erroneous knowledge into adversarial documents, and injects them into the knowledge base to attack RAG systems. The key idea of our approach is that adversarial documents, by simulating the chain-of-thought patterns aligned with the model's training signals, may be misinterpreted by the model as authentic historical reasoning processes, thus increasing their likelihood of being referenced. Experiments conducted on the MS MARCO passage ranking dataset demonstrate the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。