构建首个支持多文档类型与灵活模式的结构化信息抽取基准
ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images
- 融合实体、关系与视觉问答的统一抽取框架
- 涵盖多样文档类型，支持复杂查询与答案定位挑战
- 适合研究通用视觉语言模型在文档理解中的泛化能力
企业文档如表单和报告包含下游应用所需的关键信息，如数据归档、自动化工作流与分析。尽管通用视觉语言模型（VLMs）在现有文档理解基准上表现良好，但其在跨多种文档类型与灵活模式下进行整体性、细粒度结构化提取的能力尚未得到充分研究。现有关键实体抽取（KEE）、关系抽取（RE）与视觉问答（VQA）数据集受限于狭窄的实体本体、简单查询或同质文档类型，常忽略可适配结构化提取的需求。为此，我们提出 ExStrucTiny，一个全新的文档图像结构化信息抽取基准，整合 KEE、RE 与 VQA 的特性。该数据集通过结合人工与合成的、经人类验证的样本构建，覆盖更广泛的文档类型与抽取场景。我们在该基准上评估了开放与封闭式 VLMs，揭示了模式适应、查询不明确与答案定位等核心挑战。期望本工作为提升通用模型在文档结构化信息抽取方面的性能奠定基础。
原文摘要 · Abstract (English)
Enterprise documents, such as forms and reports, embed critical information for downstream applications like data archiving, automated workflows, and analytics. Although generalist Vision Language Models (VLMs) perform well on established document understanding benchmarks, their ability to conduct holistic, fine-grained structured extraction across diverse document types and flexible schemas is not well studied. Existing Key Entity Extraction (KEE), Relation Extraction (RE), and Visual Question Answering (VQA) datasets are limited by narrow entity ontologies, simple queries, or homogeneous document types, often overlooking the need for adaptable and structured extraction. To address these gaps, we introduce ExStrucTiny, a new benchmark dataset for structured Information Extraction (IE) from document images, unifying aspects of KEE, RE, and VQA. Built through a novel pipeline combining manual and synthetic human-validated samples, ExStrucTiny covers more varied document types and extraction scenarios. We analyze open and closed VLMs on this benchmark, highlighting challenges such as schema adaptation, query under-specification, and answer localization. We hope our work provides a bedrock for improving generalist models for structured IE in documents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。