用空间感知Transformer高效提取财务文档中的表格与键值对
Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction in Financial Documents at Scale
- 引入空间嵌入增强Transformer，同时处理文本与布局信息
- 在真实财务文档上实现高精度表格和键值对提取
- 适合需要自动化处理大量财务文件的业务场景
从财务文档中提取表格和键值对对审计、数据分析和自动发票处理等业务流程至关重要。本文提出Spatial ModernBERT——一种融合空间嵌入的Transformer模型，用于准确检测和提取复杂财务文档中的表格数据与键值字段。将抽取任务建模为三个分类头的令牌分类：(1) 标签头，将每个令牌分类为标签（如采购单号、采购日期、商品描述、数量、基础成本、建议零售价等）；(2) 列头，预测列索引；(3) 行头，区分条目行与标题行。模型在PubTables-1M数据集上预训练，再在财务文档数据集上微调，通过各分类头的交叉熵损失实现稳健性能。提出后处理方法，采用B-I-IB标注合并令牌，重建表格布局并提取键值对。实证评估表明，Spatial ModernBERT有效结合文本与空间线索，在真实财务文档中实现高精度提取。
原文摘要 · Abstract (English)
Extracting tables and key-value pairs from financial documents is essential for business workflows such as auditing, data analytics, and automated invoice processing. In this work, we introduce Spatial ModernBERT-a transformer-based model augmented with spatial embeddings-to accurately detect and extract tabular data and key-value fields from complex financial documents. We cast the extraction task as token classification across three heads: (1) Label Head, classifying each token as a label (e.g., PO Number, PO Date, Item Description, Quantity, Base Cost, MRP, etc.); (2) Column Head, predicting column indices; (3) Row Head, distinguishing the start of item rows and header rows. The model is pretrained on the PubTables-1M dataset, then fine-tuned on a financial document dataset, achieving robust performance through cross-entropy loss on each classification head. We propose a post-processing method to merge tokens using B-I-IB tagging, reconstruct the tabular layout, and extract key-value pairs. Empirical evaluation shows that Spatial ModernBERT effectively leverages both textual and spatial cues, facilitating highly accurate table and key-value extraction in real-world financial documents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。