arXiv:2607.03447cs.IRcs.AI2026-07

为自动构建的知识图谱提供分阶段评估框架,精准定位生成失败环节。

TRIAGE: Trustworthy Retrieval Instrumentation And Graph Evaluation

  • 分三阶段注入可独立解释的指标:抽取、验证、使用。
  • 无需标注数据即可诊断,关键指标如检索覆盖率与忠实性均能计算。
  • 故障定位到具体环节,指导优化抽取、图结构或检索策略。

支撑基于图的检索增强生成(Graph-RAG)的知识图谱正越来越多地由大模型自动提取构建,而非人工精修。有效评估需贯穿抽取、图构建和推理全过程,以实现故障定位,避免仅在最终答案错误时才发现问题。本文提出TRIAGE——一个面向自动化、文档驱动的Graph-RAG的阶段感知评估框架,不仅判断图是否可信,还评估其查询成本。该框架为三个阶段分别配备特定指标:知识图谱实现阶段(三元组置信度、来源覆盖度、模式与标准化检查)、专家验证阶段(图结构质量,正确性与完整性仅在有参考时离线计算)、使用阶段(检索覆盖率、忠实性、检索成本)。部署指标无需黄金标注,仅用于离线校准的指标才需要。使用时,这些指标构成必要条件诊断链,首个失效环节即定位故障源,并对应至可调节的阶段控制变量:抽取、图与模式、或检索。TRIAGE是一个理论框架,附带概念验证与可复现的评估协议。

原文摘要 · Abstract (English)

Knowledge graphs (KGs) that underpin Graph-based Retrieval-Augmented Generation (Graph-RAG) are increasingly built automatically by LLM-driven extraction rather than curated by experts. Proper evaluation would require instrumenting all pertinent stages: extraction, graph construction, and inference, coherently enough to localize failures, so that a failure at one stage is not discovered as a wrong answer at the end. We introduce TRIAGE, a stage-aware instrumentation framework for automated, document-grounded graph-RAG that asks not only whether the underlying graph can be trusted but at what cost it can be queried. TRIAGE attaches stage-specific, independently interpretable metrics to three stages: the KG Implementation (triple confidence, source coverage, and schema and canonicalization checks), the KG Validation by expert (graph-level structural quality, with correctness and completeness computed only as offline calibration when a reference is available), and the KG Usage (retrieval coverage, faithfulness, and retrieval cost); the deployed metrics need no gold annotations, the gold-requiring ones serving only as offline calibration. At usage time these metrics form a diagnostic chain of necessary conditions whose first broken link localizes the failure, and the diagnosis maps to the stage levers that can remedy it: extraction, graph and schema, or retrieval. TRIAGE is a theoretical framework with a proof of concept and a reproducible evaluation protocol.

知识图谱评估框架RAG诊断链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。