攻击知识图谱诱导AI错误推理，实证表明99%模型会信任伪造信息。
Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning
- 通过污染动态查询的知识图谱，让AI在正确推理下得出错误结论
- 30个模型中269次测试均100%信任伪造安全声明（270次中）
- 攻击效果受传递方式影响，实测比静态检测更易触发信任
我们定义了Oracle Poisoning，一种攻击类型：对手通过篡改AI代理运行时通过工具调用的结构化知识图谱，导致其基于正确推理得出错误结论。不同于提示注入，该攻击操纵的是代理所依赖的数据而非指令。我们在一个包含4200万节点的生产级代码知识图谱上验证了六种攻击场景，首次实现了对大规模智能体系统的知识图谱污染实证研究，区别于传统的嵌入式污染。主要评估基于真实SDK工具调用，涵盖来自三个供应商的九个模型（每模型30次），模型自主调用图谱查询工具并依据结果推理。结果显示：在中等攻击复杂度（L2）下，所有测试模型对污染数据的信任度达100%，269/270次试验接受伪造的安全性声明；在开放式提示下，信任度降至3%-55%，证实提示设计是重要干扰因素。攻击复杂度梯度揭示离散断点——存在最小技能阈值使信任从0%跃升至100%，将攻击问题重新定义为‘程度’而非‘是否发生’。控制实验表明，内联评估会产生假阴性：GPT-5.1在内联时信任度为0%，但在模拟与真实代理调用下均为100%，证明传递模式是一阶混淆因素。我们评估了五种防御方案；仅只读访问控制能完全消除直接篡改路径，其余四种均部分有效且依赖模型。对四个额外平台的分析表明该攻击可能在知识图谱生态系统中泛化。
原文摘要 · Abstract (English)
We define Oracle Poisoning, an attack class in which an adversary corrupts a structured knowledge graph that AI agents query at runtime via tool-use protocols, causing incorrect conclusions through correct reasoning. Unlike prompt injection, Oracle Poisoning manipulates the data agents reason over, not their instructions. We demonstrate six attack scenarios against a production 42-million-node code knowledge graph, providing the first empirical demonstration of knowledge graph poisoning against a production-scale agentic system, distinct from CTI embedding poisoning. Primary evaluation uses real SDK tool-use across nine models from three providers (N=30 per model), where models autonomously invoke a graph query tool and reason from results. The result is unambiguous: every tested model trusts poisoned data at 100% at moderate attacker sophistication(L2), with 269 valid trials (of 270) accepting fabricated security claims under directed queries. Under open-ended prompts, trust drops to 3-55%, confirming prompt framing as a confound; we report both conditions. An attacker sophistication gradient reveals discrete break points, a minimum skill at which trust flips from 0% to 100%, reframing the attack as a question not of whether but of how much. A controlled delivery-mode comparison shows that inline evaluation produces false negatives: GPT-5.1 shows 0% trust inline but 100% under both simulated and real agentic tool-use, demonstrating that delivery mode is a first-order confound. We evaluate five defences; read-only access control eliminates the direct mutation vector, while the remaining four are partial and model-dependent. Analysis of four additional platforms suggests the attack may generalise across the knowledge-graph ecosystem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。