构建可验证的文档转数据库基准,评估大模型关系忠实度。
Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction

- 通过可控的数据库生成文档流程构建真实感数据集。
- 含7341行、41935个单元格,覆盖203篇长文档与42种模式。
- 适合评估大模型在跨表关系理解与数据库完整性上的表现。
实际人工智能系统越来越需要将长而异构的文档转化为可查询的关系型数据库,而非孤立的表格。在金融、医疗、教育、交通和企业运营等领域,下游工作流依赖于标准化的模式、实体标识、主键、跨表关系和完整性约束,以支持分析、合规、审计和基于SQL的决策。现有文档转表格的基准不足以满足这一需求:将证据扁平化为单张表格会重复实体、掩盖多对多关系、产生稀疏记录,且无法检验提取的事实是否构成合法的数据库实例。这迫切需要将文档理解评价从字段提取转向数据库构建。我们提出Doc2DB-Bench,一个文档转数据库构建的基准,包含203个长文档实例,涵盖42个模式和七个领域组,共117个实体表、132个关系表、7,341行和41,935个单元格。该数据集通过可控的数据库到文档合成流程构建,并按表内提取与表间推理的分类体系组织,经真实性验证后与真实参考文献难以区分。该基准为可靠、可审计、关系忠实的大语言模型数据系统提供了测试平台。基准已公开于 https://github.com/SetonLiang/Doc2DB-Bench。
原文摘要 · Abstract (English)
Practical AI systems increasingly need to turn long, heterogeneous documents into queryable relational databases, not isolated spreadsheets. In domains such as finance, healthcare, education, transportation, and enterprise operations, downstream workflows rely on normalized schemas, entity identities, keys, cross-table relationships, and integrity constraints for analytics, compliance, auditing, and SQL-backed decision making. Existing Document-to-Table benchmarks are insufficient for this setting: flattening evidence into single tables can duplicate entities, obscure many-to-many relationships, create sparse records, and avoid testing whether extracted facts form a valid database instance. This creates an urgent need to evaluate document understanding as database construction rather than field extraction. We introduce Doc2DB-Bench, a benchmark for Document-to-Database construction, containing 203 long-document instances across 42 schemas and seven domain groups, with 117 entity tables, 132 relationship tables, 7,341 rows, and 41,935 cells. Built through a controllable DB-to-Doc synthesis pipeline and organized by a taxonomy of intra-table extraction and inter-table reasoning, the generated documents undergo authenticity verification, proving indistinguishable from real-world references. Doc2DB-Bench thus provides a testbed for reliable, auditable, and relationally faithful LLM-based data systems. The benchmark is publicly available at https://github.com/SetonLiang/Doc2DB-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。