只需改几个词就能骗过图结构检索生成系统,让大模型答错。
A Few Words Can Distort Graphs: Knowledge Poisoning Attacks on Graph-based Retrieval-Augmented Generation of Large Language Models
- 用图论分析找弱点节点,精准修改相关文字操纵答案。
- 改少于0.05%的文本,问答准确率从95%降到50%。
- 现有防御手段无效,适合关注模型安全的研究者看。
基于图的检索增强生成(GraphRAG)通过将原始文本转化为结构化知识图谱,提升了大语言模型(LLM)的准确性与可解释性。然而,GraphRAG在构建图谱时依赖LLM从文本中提取知识,这一过程可能被恶意篡改以植入误导信息。我们提出两种知识投毒攻击(KPAs),证明仅修改源文本中的少数词汇即可显著改变生成图谱,毒害GraphRAG并严重误导下游推理。第一种目标型攻击(TKPA)利用图论分析定位脆弱节点,通过LLM重写对应叙述,在93.1%成功率下实现对特定问答结果的精准控制,且投毒文本流畅自然。第二种通用型攻击(UKPA)则通过修改影响全局的关键词(如代词、依存关系),破坏图谱结构完整性,仅需改动不足0.05%的全文,问答准确率即从95%暴跌至50%。实验表明,当前最先进的防御方法无法检测这些攻击,凸显出对GraphRAG管道进行知识投毒防护仍处于未探索状态。
原文摘要 · Abstract (English)
Graph-based Retrieval-Augmented Generation (GraphRAG) has recently emerged as a promising paradigm for enhancing large language models (LLMs) by converting raw text into structured knowledge graphs, improving both accuracy and explainability. However, GraphRAG relies on LLMs to extract knowledge from raw text during graph construction, and this process can be maliciously manipulated to implant misleading information. Targeting this attack surface, we propose two knowledge poisoning attacks (KPAs) and demonstrate that modifying only a few words in the source text can significantly change the constructed graph, poison the GraphRAG, and severely mislead downstream reasoning. The first attack, named Targeted KPA (TKPA), utilizes graph-theoretic analysis to locate vulnerable nodes in the generated graphs and rewrites the corresponding narratives with LLMs, achieving precise control over specific question-answering (QA) outcomes with a success rate of 93.1\%, while keeping the poisoned text fluent and natural. The second attack, named Universal KPA (UKPA), exploits linguistic cues such as pronouns and dependency relations to disrupt the structural integrity of the generated graph by altering globally influential words. With fewer than 0.05\% of full text modified, the QA accuracy collapses from 95\% to 50\%. Furthermore, experiments show that state-of-the-art defense methods fail to detect these attacks, highlighting that securing GraphRAG pipelines against knowledge poisoning remains largely unexplored.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。