arXiv:2608.10679cs.IRcs.AI2026-08被引 1

构建企业问答中隐含组织关系推理的新基准,挑战模型从文档中还原隐藏逻辑的能力。

ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answering

论文配图:ENTLORE: A Graph-Grounded Benchmark for Latent Organizational Reasoning in Enterprise Question Answering
图 1 · 摘自论文原文
  • 基于企业文档与组织图谱重建真实业务世界,生成带验证关系的真相图
  • 隐含关系推理任务仍有30.4%未解答,远高于显式查询的12.6%
  • 适合评估大模型在企业知识系统中挖掘深层逻辑的能力

企业问答依赖内部文档检索与生成答案,但日常记录中的组织关系常隐含于异构数据中。现有基准多预设答案路径,仅测试事实拼接而非隐含关系推断。本文提出ENTLORE,一个图结构驱动的基准构建框架,通过审计的企业文档、权威组织表和运营记录重建可验证的企业世界。版本化组织规范确保关系真实性,并生成完整答案与证明凭证。匿名发布仅提供文档集合,隐藏结构与目标关系。该数据集包含2,341份文档(三类来源)与907个问题,涵盖显式查找、跨源组合及隐含组织推理,覆盖56种模型与访问配置。将释放世界建模为实体图或可导航知识库可得最佳效果,但即便提供黄金文档,仍有30.4%的隐含问题无法回答,显著高于显式(12.6%)与组合型(6.2%)问题。表明企业问答不仅依赖文档召回,更取决于能否激活隐含组织关系。代码与数据已开源。

原文摘要 · Abstract (English)

Enterprise question answering is framed as retrieving internal documents and generating grounded answers. Routine enterprise records, however, are work by-products in which required organizational relations remain implicit across heterogeneous sources. Existing benchmarks provide realistic multi-source evidence, but often materialize a predefined answer path and therefore test the composition of stated facts rather than recovery of a target relation absent from the corpus. We call the latter capability latent organizational reasoning. We introduce ENTLORE, a graph-grounded benchmark construction framework that reconstructs an audited enterprise world from routine documents, authoritative organizational tables, and operational records. Versioned organizational conventions certify derived relations in a truth graph, enabling complete golden answers and proof certificates. The aligned anonymized release exposes only the document corpus while withholding private structure and target relations. ENTLORE contains 2,341 documents from three source types and 907 questions spanning explicit lookup, cross-source composition, and latent organizational reasoning, evaluated across 56 model and access configurations. Structuring the released world as an induced entity graph or navigable knowledge base gives the strongest deployable results. Yet supplying gold documents still leaves 30.4% of latent questions unanswered, versus 12.6% and 6.2% for explicit and compositional questions. Enterprise QA therefore depends not only on document recall, but also on whether implicit organizational relations become usable. The benchmark, data, and code are publicly available at https://github.com/scitix/entlore .

企业问答关系推理知识图谱评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。