通过解耦语言信息，实现多语言文档理解的跨语言泛化。
LDP: Generalizing to Multilingual Visual Information Extraction by Language Decoupled Pretraining
- 用扩散模型剥离图像中的语言特征，仅保留视觉与版式信息。
- 在多语言数据上训练后，跨语言性能显著超越现有模型。
- 适合需要支持多种语言文档解析的应用场景。
视觉信息抽取（VIE）在理解半结构化文档中起关键作用，已有大量预训练模型提升性能，但多数为单语种（通常为英语）。由于英语与其他语言之间预训练语料数量和质量极度不平衡，少有工作能有效扩展到非英语场景。本文通过系统实验发现，不同语言图像中的视觉与版式模态具有不变性。若将语言偏差从文档图像中解耦，基于视觉-版式信息的模型可实现出色的跨语言泛化能力。为此，我们提出一种简单有效的多语言训练范式LDP（Language Decoupled Pretraining），用于更好利用单语种预训练数据。所提模型LDM（Language Decoupled Model）首先在语言无关数据上预训练，通过扩散模型解耦语言知识，随后在下游语言上微调。大量实验证明，LDM优于所有现有SOTA多语言预训练模型，并在单语/英语下游任务上仍具竞争力。
原文摘要 · Abstract (English)
Visual Information Extraction (VIE) plays a crucial role in the comprehension of semi-structured documents, and several pre-trained models have been developed to enhance performance. However, most of these works are monolingual (usually English). Due to the extremely unbalanced quantity and quality of pre-training corpora between English and other languages, few works can extend to non-English scenarios. In this paper, we conduct systematic experiments to show that vision and layout modality hold invariance among images with different languages. If decoupling language bias from document images, a vision-layout-based model can achieve impressive cross-lingual generalization. Accordingly, we present a simple but effective multilingual training paradigm LDP (Language Decoupled Pre-training) for better utilization of monolingual pre-training data. Our proposed model LDM (Language Decoupled Model) is first pre-trained on the language-independent data, where the language knowledge is decoupled by a diffusion model, and then the LDM is fine-tuned on the downstream languages. Extensive experiments show that the LDM outperformed all SOTA multilingual pre-trained models, and also maintains competitiveness on downstream monolingual/English benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。