单文档投毒攻击让大模型推理失效，暴露RAG系统安全漏洞
AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning
- 针对RAG系统，仅污染一个文档即可诱导大模型错误推理
- 通过与模型交互迭代优化，使攻击更隐蔽且效果显著
- 揭示大模型推理链中的脆弱点，适合安全研究者参考
检索增强生成（RAG）通过引入外部文档提升大语言模型（LLM）的推理能力，但也带来新的攻击面。本文研究了RAG中的知识库投毒攻击：攻击者向检索语料库注入恶意内容，使其被检索器自然召回并被LLM在推理时使用。不同于以往大规模污染语料库的方法，本文提出AdversarialCoT——一种查询特定的单文档投毒攻击。该方法首先提取目标LLM的推理框架，构建初始对抗性思维链（CoT），随后通过与LLM的多次交互，逐步优化恶意文档，暴露出并利用关键推理漏洞。在基准大模型上的实验表明，仅一个恶意文档即可显著降低推理准确率，揭示出细微但影响深远的安全弱点。本研究揭示了RAG系统的潜在风险，并为设计更鲁棒的LLM推理流程提供了实践洞见。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) enhances large language model (LLM) reasoning by retrieving external documents, but also opens up new attack surfaces. We study knowledge-base poisoning attacks in RAG, where an attacker injects malicious content into the retrieval corpus, which is then naturally surfaced by the retriever and consumed by the LLM during reasoning. Unlike prior work that floods the corpus with poisoned documents, we propose AdversarialCoT, a query-specific attack that poisons only a single document in the corpus. AdversarialCoT first extracts the target LLM's reasoning framework to guide the construction of an initial adversarial chain-of-thought (CoT). The adversarial document is iteratively refined through interactions with the LLM, progressively exposing and exploiting critical reasoning vulnerabilities. Experiments on benchmark LLMs show that a single adversarial document can significantly degrade reasoning accuracy, revealing subtle yet impactful weaknesses. This study exposes security risks in RAG systems and provides actionable insights for designing more robust LLM reasoning pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。