arXiv:2412.16708cs.IR2024-12被引 17

研究如何让RAG系统抵抗恶意数据攻击,提升生成可靠性。

Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks

  • 区分三类上下文:恶意、原始和引导型,分析其对输出影响。
  • 发现'怀疑式提示'可部分激活模型自检能力,但依赖推理水平。
  • 评测多个检索器,揭示其暴露模型于攻击的风险差异。

检索增强生成(RAG)系统被视作缓解大模型幻觉、提升知识密集型任务性能的有前景方案。然而,这类系统易受对抗性投毒攻击:恶意段落注入检索库后,可误导模型生成事实错误内容。本文通过严格控制的实证研究,系统分析RAG在攻击下的表现及鲁棒性提升路径。生成端提出上下文类型分类体系——对抗性、未触达和引导型,并分析其单独与联合效应;检索端评估多个检索器,衡量其暴露大模型于对抗上下文的程度。研究发现,'怀疑式提示'可激发模型内部推理机制,实现部分自我防御,但效果高度依赖模型自身推理能力。实验代码已公开(https://github.com/JinyanSu1/eval_PoisonRaG),为构建更安全、可靠的RAG系统提供可操作洞见,助力真实场景落地。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) systems have emerged as a promising solution to mitigate LLM hallucinations and enhance their performance in knowledge-intensive domains. However, these systems are vulnerable to adversarial poisoning attacks, where malicious passages injected into the retrieval corpus can mislead models into producing factually incorrect outputs. In this paper, we present a rigorously controlled empirical study of how RAG systems behave under such attacks and how their robustness can be improved. On the generation side, we introduce a structured taxonomy of context types-adversarial, untouched, and guiding-and systematically analyze their individual and combined effects on model outputs. On the retrieval side, we evaluate several retrievers to measure how easily they expose LLMs to adversarial contexts. Our findings also reveal that "skeptical prompting" can activate LLMs' internal reasoning, enabling partial self-defense against adversarial passages, though its effectiveness depends strongly on the model's reasoning capacity. Together, our experiments (code available at https://github.com/JinyanSu1/eval_PoisonRaG) and analysis provide actionable insights for designing safer and more resilient RAG systems, paving the way for more reliable real-world deployments.

RAG对抗攻击模型鲁棒性生成安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。