通过分析注意力权重，检测RAG系统中的隐蔽毒化攻击。
Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAG
- 利用注意力分数识别被污染段落的异常影响。
- 新方法使防御准确率比基线高约20%。
- 适合关注RAG安全性的研究者与开发者。
检索增强生成（RAG）系统易受毒化攻击，即使在低污染率下也存在风险。现有攻击缺乏隐蔽性，可被可靠检测。本文提出基于可区分性的安全博弈框架，量化攻击隐蔽性；若少数污染段落主导输出，则必然显著干扰推理过程，破坏隐蔽性。为此，我们分析大模型中间信号如注意力权重，以近似各段落对输出的影响。提出归一化段落注意力得分（NPAS）和轻量级注意力方差过滤器（AV Filter），用于标记异常段落。该方法提升系统鲁棒性，使防御准确率相较基线最高提升约20%。同时构建自适应攻击以掩盖异常，成功率达35%，凸显实现真正隐蔽毒化的挑战。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems are vulnerable to attacks that inject poisoned passages into the retrieved context, even at low corruption rates. We show that existing attacks are not designed to be stealthy, allowing reliable detection and mitigation. We formalize a distinguishability-based security game to quantify stealth for such attacks. If a few poisoned passages control the response, they must bias the inference process more than the benign ones, inherently compromising stealth. This motivates analyzing intermediate signals of LLMs, such as attention weights, to approximate the influence of different passages on the response. Leveraging attention weights, we introduce the $\textbf{Normalized Passage Attention Score}$ (NPAS) and a lightweight $\textbf{Attention-Variance Filter}$ (AV Filter) that flags anomalous passages. Our method improves robustness, yielding up to $\sim$ $\textbf{20%}$ higher accuracy than baseline defenses. We also develop adaptive attacks that attempt to conceal such anomalies, achieving up to $\textbf{35%}$ success rate and underscoring the challenges of achieving true stealth in poisoning RAG systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。