arXiv:2607.11562cs.CV2026-07被引 1

专为文档设计的视觉文本模型,显著提升多语言文档理解与解析能力。

MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

论文配图:MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
图 1 · 摘自论文原文
  • 构建1.13亿图像的多语言文档预训练数据集,支持17种语言
  • 联合图文生成与像素级文档重建,保留字符笔画和版式细节
  • 可作为轻量级模型的视觉编码器,性能超越现有开源方案

主流视觉编码器基于自然图像预训练,难以直接用于文档图像分析,因密集文本与精细字符笔画需要字符级视觉感知。本文提出MonkeyOCRv2,一种面向文档智能的视觉-文本预训练模型。首先,构建了目前最大规模的文档图像预训练语料库MonkeyDoc v2,包含1.13亿张跨越17种语言的图像。其次,提出一种联合预训练策略,同时学习图像到文本生成与像素级文档重建:前者对齐视觉表征与文本内容,后者保留字符笔画与版式细节。在五个代表性文档分析任务上进行实验,包括文本识别、公式识别、文本检测、文档篡改检测和重叠文本分割。将原编码器替换为MonkeyOCRv2后,所有任务性能均一致提升。最后,验证其作为多模态大模型视觉编码器的有效性,在更复杂的文档解析与理解任务中表现优异。保持冻结状态并搭配轻量语言模型,得到一个0.7B参数的文档解析模型,在覆盖数字生成与拍摄文档的17种语言的MDPBench基准上达到新的开源SOTA,相比此前最佳的3B dots.mocr提升2.8个百分点,且视觉编码器仅为其约1/11大小。该冻结编码器还驱动了一个文档理解模型,在八个基准上超越基于CLIP、DINO和SAM的同类模型,且在相同训练设置下表现更优。结果表明,面向文档的视觉预训练可独立成为文档智能的基础。

原文摘要 · Abstract (English)

Mainstream visual encoders are pretrained on natural images and cannot be effectively applied to document images without document-oriented adaptation, as dense text and fine-grained character strokes demand character-level visual perception. We present MonkeyOCRv2, a visual-text pretrained model for document AI. First, we construct MonkeyDoc v2, to our knowledge the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages. Second, we propose a pretraining strategy that jointly learns image-to-text generation and pixel-level document reconstruction: the former aligns visual representations with textual content, while the latter preserves character strokes and layout details. Extensive experiments are conducted on five representative document analysis tasks, including text recognition, formula recognition, text detection, document tampering detection, and overlapping text segmentation. Replacing the original encoders with MonkeyOCRv2 consistently improves performance across all five tasks. Finally, we validate its effectiveness as the vision encoder of multimodal large language models on the more challenging tasks of document parsing and document understanding. Kept frozen and paired with a lightweight language model, it yields a 0.7B document parsing model that sets a new open-source state-of-the-art on MDPBench, a recent benchmark spanning digital-born and photographed documents across 17 languages, surpassing the previous best 3B dots.mocr by 2.8% absolute with a vision encoder roughly 11$\times$ smaller. The frozen encoder also powers a document understanding model that outperforms counterparts built on CLIP, DINO, and SAM across eight benchmarks under identical training settings. These results suggest that document-oriented visual pretraining can serve as a foundation for document intelligence in its own right.

文档智能视觉预训练多语言轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。