用强化学习提升扫描文档布局理解,通用性更强。
Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
- 基于复合奖励的强化学习框架,优化布局解析
- 在4个基准上超越现有模型,跨领域表现优异
- 开源数据集与模型,助力可复现研究
从扫描图像中解析文档为结构化格式仍具挑战,因文本段落、图表、公式和表格等元素交织复杂。现有监督微调方法在不同文档类型间泛化能力差,尤其在分布外数据上表现不佳,且高质量布局感知训练数据稀缺。为此,我们提出LayoutRL强化学习框架,通过归一化编辑距离、段落数准确率和阅读顺序保持的复合奖励优化布局理解。为支持训练,构建了Infinity-Doc-400K数据集,并在此基础上训练出Infinity-Parser视觉语言模型,在OmniDocBench、olmOCR-Bench、PubTabNet和FinTabNet等多个基准上持续取得最佳性能,显著优于专用文档解析系统和通用视觉语言模型。我们将公开代码、数据集与模型,推动文档解析领域可复现研究。
原文摘要 · Abstract (English)
Document parsing from scanned images into structured formats remains a significant challenge due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables. Existing supervised fine-tuning methods often struggle to generalize across diverse document types, leading to poor performance, particularly on out-of-distribution data. This issue is further exacerbated by the limited availability of high-quality training data for layout-aware parsing tasks. To address these challenges, we introduce LayoutRL, a reinforcement learning framework that optimizes layout understanding through composite rewards integrating normalized edit distance, paragraph count accuracy, and reading order preservation. To support this training, we construct the Infinity-Doc-400K dataset, which we use to train Infinity-Parser, a vision-language model demonstrating robust generalization across various domains. Extensive evaluations on benchmarks including OmniDocBench, olmOCR-Bench, PubTabNet, and FinTabNet show that Infinity-Parser consistently achieves state-of-the-art performance across a broad range of document types, languages, and structural complexities, substantially outperforming both specialized document parsing systems and general-purpose vision-language models. We will release our code, dataset, and model to facilitate reproducible research in document parsing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。