构建真实票据理解新基准，提升多模态模型推理能力
From Recognition to Reasoning: Benchmarking and Enhancing MLLMs on Real-World Receipt Document Understanding
- 分四层任务评估模型从识别到推理的全流程能力
- 新方法在复杂推理任务上超越主流商用模型
- 适合研究文档理解与多模态推理的学者使用
从视觉文档中提取结构化信息（视觉信息抽取，VIE）是企业自动化的核心。尽管近期多模态大模型（MLLMs）展现出潜力，但现有基准在规模、真实性、语义粒度和文档类型覆盖上存在明显不足。为此，我们提出ReceiptBench，一个包含10,000张多样化票据的大规模人工标注基准，将信息抽取划分为四个层级子任务：(1) 基础感知（原始文本定位），(2) 格式标准化（严格遵循规范指令），(3) 语义推理（从上下文推断隐含属性），(4) 结构解析（处理嵌套条目）。此外，我们提出两阶段训练框架，结合度量感知的组相对策略优化（GRPO），将严格的评估约束转化为强化学习信号，增强结构一致性。大量实验表明，该方法在复杂推理任务上达到当前最优表现，超越领先专有模型。数据集与代码已开源：https://github.com/wwwT0ri/ReceiptBench。
原文摘要 · Abstract (English)
Extracting structured information from visual documents (Visual Information Extraction, VIE) is a cornerstone of business automation. While recent Multimodal Large Language Models (MLLMs) have shown promising capabilities, existing benchmarks suffer from critical limitations in scale and realism, lack semantic granularity, and fail to cover diverse document types. To bridge this gap, we introduce ReceiptBench, a large-scale, human-annotated benchmark consisting of 10k diverse receipts, organizing information extraction into four hierarchical sub-tasks: (1) Basic Perception for raw text spotting, (2) Format Normalization for strictly following standardization instructions, (3) Semantic Reasoning for inferring implicit attributes from context, and (4) Structure Parsing for handling nested line items. Furthermore, we propose a two-stage training framework incorporating Metric-Aware Group Relative Policy Optimization (GRPO), which translates rigorous evaluation constraints into reinforcement learning signals to enhance structural consistency. Extensive experiments demonstrate that our method yields state-of-the-art performance, surpassing leading proprietary models on complex reasoning tasks. We release our datasets and code at https://github.com/wwwT0ri/ReceiptBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。