arXiv:2605.00460cs.CRcs.LG2026-05被引 1

通过相似性图谱识别知识库中恶意文档,防止提示注入攻击。

CleanBase: Detecting Malicious Documents in RAG Knowledge Databases

论文配图:CleanBase: Detecting Malicious Documents in RAG Knowledge Databases
图 1 · 摘自论文原文
  • 构建文档语义相似性图,检测高相似度异常集群。
  • 在多个数据集上实现接近100%的恶意文档检出率。
  • 适合关注RAG系统安全的开发者与研究者使用。

检索增强生成(RAG)易受提示注入攻击,攻击者将精心设计的恶意文档注入知识库。当用户提出针对性问题时,RAG系统可能召回这些文档,其嵌入的恶意提示会诱导模型生成攻击者指定的答案,破坏系统完整性。本文提出CleanBase,通过分析同一攻击目标下恶意文档的高语义相似性特征,构建基于嵌入模型的相似性图谱。若两文档语义相似度超过统计阈值,则建立边连接;由于恶意文档具有内在一致性,通常形成紧密子图(团)。CleanBase检测此类团并标记相关文档为恶意。理论上推导了误报与漏报的上界,并在多数据集及多种攻击场景下验证了其有效性。实验表明,该方法能精准检测恶意文档,有效保护RAG系统。代码已开源:https://github.com/WeifeiJin/CleanBase。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) is vulnerable to prompt injection attacks, in which an adversary inserts malicious documents containing carefully crafted injected prompts into the knowledge database. When a user issues a question targeted by the attack, the RAG system may retrieve these malicious documents, whose injected prompts mislead it into generating attacker-specified answers, thereby compromising the integrity of the RAG system. In this work, we propose CleanBase, a method to detect malicious documents within a knowledge database. Our key insight is that malicious documents crafted for the same attack-targeted questions often exhibit high semantic similarity, as attackers deliberately make them consistent to improve attack success rates. Accordingly, CleanBase constructs a similarity graph over the knowledge database, where each node represents a document and an edge connects two nodes if their semantic similarity--computed using an embedding model--exceeds a statistically determined threshold. Due to their inherent similarity, malicious documents tend to form cliques within this graph. CleanBase detects such cliques and flags the corresponding documents as malicious. We theoretically derive upper bounds on CleanBase's false positive and false negative rates and empirically validate its effectiveness. Experimental results across multiple datasets and prompt injection attacks demonstrate that CleanBase accurately detects malicious documents and effectively safeguards RAG systems. Our source code is available at https://github.com/WeifeiJin/CleanBase.

RAG安全恶意检测提示注入图分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。