arXiv:2602.02053cs.CL2026-02ACL被引 2

构建真实场景下的图谱增强生成评测基准,检验系统在长文本与异构文档中的表现。

WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora

  • 基于维基百科的引用结构构建多层级知识图谱,模拟真实检索场景。
  • 包含1100个问题,覆盖单事实、多事实问答及章节级摘要三类复杂度。
  • 揭示当前图谱生成模型在细节捕捉上的不足,尤其在摘要任务中表现欠佳。

基于图谱的检索增强生成(GraphRAG)将外部知识组织为层次化图谱,实现跨多文档的高效检索与证据聚合。然而,现有GraphRAG评测大多依赖短篇精选片段作为外部知识,难以评估系统在真实场景下处理长上下文与大规模异构文档的能力。为此,我们提出WildGraphBench,一个面向真实世界应用的GraphRAG评测基准。利用维基百科独特的结构——连贯叙述依托于长且异质的外部参考文献,我们选取12个顶级主题的文章,以它们的外部引用作为检索语料库,以引用链接的陈述作为真实答案,构建了包含1,100个问题的数据集,涵盖单事实问答、多事实问答和章节级摘要三个难度层级。在多个基线模型上的实验表明,当前GraphRAG流水线在证据来自中等数量来源时有助于多事实聚合,但该范式可能过度强调高层级陈述而忽略细粒度细节,导致在摘要任务上表现较差。

原文摘要 · Abstract (English)

Graph-based Retrieval-Augmented Generation (GraphRAG) organizes external knowledge as a hierarchical graph, enabling efficient retrieval and aggregation of scattered evidence across multiple documents. However, many existing benchmarks for GraphRAG rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. To bridge this gap, we introduce WildGraphBench, a benchmark designed to assess GraphRAG performance in the wild. We leverage Wikipedia's unique structure, where cohesive narratives are grounded in long and heterogeneous external reference documents, to construct a benchmark reflecting real-word scenarios. Specifically, we sample articles across 12 top-level topics, using their external references as the retrieval corpus and citation-linked statements as ground truth, resulting in 1,100 questions spanning three levels of complexity: single-fact QA, multi-fact QA, and section-level summarization. Experiments across multiple baselines reveal that current GraphRAG pipelines help on multi-fact aggregation when evidence comes from a moderate number of sources, but this aggregation paradigm may overemphasize high-level statements at the expense of fine-grained details, leading to weaker performance on summarization tasks. Project page:https://github.com/BstWPY/WildGraphBench.

图谱增强评测基准信息检索大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。