arXiv:2607.04636cs.CV2026-07

用场景感知生成文档数据,让小模型也能高效提取关键信息

Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis

  • 基于少量样本文档自动合成带标注的文档-模式-注释三元组
  • 引入错误驱动生成,让模型学会处理真实世界中的复杂错误案例
  • 特别提升小模型和开放类别场景下的提取准确率,适合本地部署

关键信息抽取(KIE)将视觉丰富的文档转化为结构化数据,但实际部署仍具挑战:高性能通常依赖昂贵的服务器级大模型,而轻量级本地模型缺乏足够的监督信号。本文提出SAYRE,一种无需手工模板设计的场景感知文档合成框架,可生成可扩展的KIE训练数据。给定少量样本文档,SAYRE捕捉类别相关的文本模式与版式规范,自动生成文档-模式-标注三元组。进一步引入错误驱动生成机制,将真实世界中的失败案例转化为具有结构性的难例训练样本。在受限与开放类别的KIE任务上实验表明,SAYRE持续提升Qwen3-VL基线模型表现,并在本地部署的大模型中达到最优整体性能。数据规模实验显示,随着合成数据增加,性能呈上升趋势,尤其对小型模型及开放类别提取效果显著。错误分析表明,合成数据训练使字段级错误减少,提升了密集表格、业务标识符和合同条款等场景下的模式感知提取能力。这些结果确立了场景感知合成作为提升实用多模态KIE的有效数据驱动方法。

原文摘要 · Abstract (English)

Key Information Extraction (KIE) converts visually rich documents into structured data, but practical deployment remains challenging: strong performance often relies on costly on-server Large Multimodal Models (LMMs), while compact locally deployable models lack sufficient KIE supervision. We present SAYRE, a scene-aware document synthesis framework for generating scalable KIE training data without hand-crafted template design. Given a few exemplar documents, SAYRE captures category-specific content patterns and layout conventions to synthesize document-schema-annotation triples. It further introduces error-driven generation, which expands real-world failure cases into hard training examples while preserving their structural patterns. Experiments on constrained- and open-category KIE show that SAYRE consistently improves Qwen3-VL backbones and achieves the strongest overall performance among on-device LMMs. Data scaling experiments show an overall upward trend as more synthesized data is introduced, especially for smaller models and open-category extraction. Error analysis further shows that synthesized training reduces field-level errors by improving schema-aware extraction over dense tables, business identifiers, and contract clauses. These results establish scene-aware synthesis as an effective data-centric approach for improving practical multimodal KIE.

多模态信息抽取数据合成本地部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。