定位RAG系统中被污染的知识来源,提升对抗攻击的可追溯性。
Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generation
- 基于检索排名、语义相关性和生成影响构建责任评分体系
- 在7个数据集15种攻击下准确识别中毒文本,鲁棒性强
- 适合安全审计、模型可信度评估等场景使用
检索增强生成(RAG)通过引入外部知识提升大语言模型输出质量,但易受投毒攻击影响,恶意文本插入知识库会误导模型生成。尽管已有多种防御措施,仍常被更自适应或复杂的攻击绕过。本文提出RAGOrigin,一种黑盒责任溯源框架,用于识别导致错误生成的知识库文本。该方法为每次误生成事件构建聚焦的溯源范围,通过评估候选文本的检索排名、语义相关性及对生成结果的影响,分配责任分数,并采用无监督聚类分离中毒文本。我们在7个数据集和15种投毒攻击(包括新设计的自适应策略与多攻击者场景)上验证该方法,其在识别中毒内容方面优于现有基线,且在动态与噪声条件下保持稳健。结果表明,RAGOrigin为追踪RAG系统中被污染知识的源头提供了实用有效的解决方案。代码已公开于:https://github.com/zhangbl6618/RAG-Responsibility-Attribution
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) integrates external knowledge into large language models to improve response quality. However, recent work has shown that RAG systems are highly vulnerable to poisoning attacks, where malicious texts are inserted into the knowledge database to influence model outputs. While several defenses have been proposed, they are often circumvented by more adaptive or sophisticated attacks. This paper presents RAGOrigin, a black-box responsibility attribution framework designed to identify which texts in the knowledge database are responsible for misleading or incorrect generations. Our method constructs a focused attribution scope tailored to each misgeneration event and assigns a responsibility score to each candidate text by evaluating its retrieval ranking, semantic relevance, and influence on the generated response. The system then isolates poisoned texts using an unsupervised clustering method. We evaluate RAGOrigin across seven datasets and fifteen poisoning attacks, including newly developed adaptive poisoning strategies and multi-attacker scenarios. Our approach outperforms existing baselines in identifying poisoned content and remains robust under dynamic and noisy conditions. These results suggest that RAGOrigin provides a practical and effective solution for tracing the origins of corrupted knowledge in RAG systems. Our code is available at: https://github.com/zhangbl6618/RAG-Responsibility-Attribution
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。