arXiv:2603.18034cs.CRcs.AI2026-03被引 1

攻击者可伪造检索文档,诱导大模型输出恶意内容。

Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems

  • 用梯度优化生成隐藏和触发文档,操控检索结果。
  • 纯向量检索下攻击成功率达38%,混合检索可降至0%。
  • 适合关注RAG安全性的研究人员与系统设计者。

检索增强生成(RAG)系统通过外部知识源扩展大语言模型能力,但检索环节引入新攻击面。攻击者可污染检索语料库,使恶意文档在推理时优先被检索,进而操纵模型输出。本文研究基于梯度的语料库投毒攻击,评估无需修改底层LLM的检索层防御机制。实现双文档投毒攻击,包含经贪心坐标梯度(GCG)优化的“睡袋文档”和“触发文档”。在含67,941篇文档的Security Stack Exchange语料库上,50次攻击测试中,纯向量检索下共现检索率达38.0%。实验表明,简单结合BM25与向量相似度的混合检索策略可将攻击成功率从38%降至0%,且无需修改模型或重训练检索器。当攻击者同时优化稀疏与稠密信号时,混合检索仍可被部分绕过(成功率20–44%),但显著提高攻击难度。跨五类大模型(GPT-5.3、GPT-4o、Claude Sonnet 4.6、Llama 4、GPT-4o-mini)评估显示,攻击成功率介于46.7%至93.3%之间。在FEVER维基百科数据集上的跨语料库测试(25次攻击)中,所有检索配置均未成功。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) systems extend large language models (LLMs) with external knowledge sources but introduce new attack surfaces through the retrieval pipeline. In particular, adversaries can poison retrieval corpora so that malicious documents are preferentially retrieved at inference time, enabling targeted manipulation of model outputs. We study gradient-guided corpus poisoning attacks against modern RAG pipelines and evaluate retrieval-layer defenses that require no modification to the underlying LLM. We implement dual-document poisoning attacks consisting of a sleeper document and a trigger document optimized using Greedy Coordinate Gradient (GCG). In a large-scale evaluation on the Security Stack Exchange corpus (67,941 documents) with 50 attack attempts, gradient-guided poisoning achieves a 38.0 percent co-retrieval rate under pure vector retrieval. We show that a simple architectural modification, hybrid retrieval combining BM25 and vector similarity, substantially mitigates this attack. Across all 50 attacks, hybrid retrieval reduces gradient-guided attack success from 38 percent to 0 percent without modifying the model or retraining the retriever. When attackers jointly optimize payloads for both sparse and dense retrieval signals, hybrid retrieval can be partially circumvented, achieving 20-44 percent success, but still significantly raises attack difficulty relative to vector-only retrieval. Evaluation across five LLM families (GPT-5.3, GPT-4o, Claude Sonnet 4.6, Llama 4, and GPT-4o-mini) shows attack success ranging from 46.7 percent to 93.3 percent. Cross-corpus evaluation on the FEVER Wikipedia dataset (25 attacks) yields 0 percent attack success across all retrieval configurations.

RAG安全投毒攻击检索增强对抗样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。