结构化知识图谱只在模型未学过的内容上有效，对已有知识无帮助。
Knowledge-Graph Grounding Helps LLMs Only for Out-of-Training Knowledge: A Controlled Study on Clinical Question Answering
- 用知识图谱与向量混合引擎，测试不同模型在医疗问答中的表现。
- 图谱仅在新知识上提升准确率（+68至+79），已有知识上无改善。
- 适用于需要外部新增医学知识的场景，如临床研究或罕见病诊断。
近期《自然·医学》研究称通用前沿大模型在医学基准上优于专业检索增强工具，且检索可能损害强模型性能。我们追问：结构化知识图谱（KG）能否改变这一现象？何时图谱才真正有用？研究得出两项结果：第一，复现发现原报告的健康评估得分（~88）实为共识子集，全集下前沿模型与理想答案均得分为46-47（医师校准评分，一致性82.5%），并复现了GPT-5.2共识得分90.9，同时发现评分器存在缺陷；第二，基于公开生物医学知识图谱PrimeKG的图谱+向量引擎（samyama-graph），无论直接三元组检索还是自然语言转Cypher的智能查询（成功率82%），均未提升MedQA在弱到强模型链上的表现（|Delta| ≤ 3.4）。但在合成反事实图谱及混合已知与未知事实的混合基准中，相同流程使模型在训练外知识上的准确率从随机水平跃升至约100%（提升68至79），而对已知事实无增益（无需大模型即可回答）。三种情形（无知识、图谱辅助、混合）表明：图谱仅当关键信息超出模型训练范围时才有价值——公共知识冗余，私有与新颖数据才是图谱发挥优势之处，呼应原始研究关于机构数据的警告。
原文摘要 · Abstract (English)
A recent Nature Medicine study reports that general-purpose frontier LLMs outperform specialized retrieval-augmented clinical tools on medical benchmarks, and that retrieval can hurt strong models. We ask the natural follow-up: does structured knowledge-graph (KG) grounding change this, and when does grounding help at all? We contribute two results. First, a reproduction: the study's headline HealthBench score (~88) is the Consensus variant, not full HealthBench, where frontier models and ideal completions both score ~46-47 under a physician-calibrated grader (agreement 82.5%); we reproduce GPT-5.2 Consensus =90.9 and flag a score-deflating grader bug. Second, a knowledge-boundary result. Using a graph+vector engine (samyama-graph) over the public biomedical KG PrimeKG, neither naive triple retrieval nor an agentic natural-language-to-Cypher loop (82% successful queries) improves MedQA across a weak-to-strong model ladder (all |Delta| <= 3.4). On a synthetic counterfactual KG, and on a hybrid benchmark mixing known and novel facts, the identical pipeline lifts out-of-training accuracy from chance to ~100% (+68 to +79) while adding nothing on known facts (a no-LLM arm answers both). Across three regimes (no-knowledge, graph-aided, hybrid), grounding helps only insofar as the decisive fact lies outside the model's training -- public-KG facts are redundant, private and novel data are where it pays -- matching the study's institutional-data caveat.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。