arXiv:2508.13238cs.CV2025-08被引 10

通过推理与工具交替机制,提升OCR模型的准确率和鲁棒性。

DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Model

  • 采用推理与专家模型交替调用的框架,增强视觉聚焦能力。
  • 在ReST和OmniDocBench上优于非推理模型与单一专家模型。
  • 适合需要高精度、低幻觉的文档识别场景。

近年来,视觉语言模型(VLMs)在端到端文档解析与理解方面取得进展,在多种光学字符识别(OCR)任务中表现优异。然而,这些模型因过度依赖语言先验,常产生图像中不存在的词汇。相比之下,传统OCR模型虽具备更强的细粒度视觉感知且幻觉较少,但缺乏上下文语义理解与推理能力,难以应对复杂情况。为此,我们提出DianJin-OCR-R1,一种基于推理与工具交替训练的增强型识别框架。该模型首先利用自身OCR能力识别图像内容,再调用其他专家模型获取参考结果;随后被引导“再次审视”图像,对比自身输出与外部结果以发现错误或遗漏;最后整合所有证据生成更精确的输出。该设计使模型学会隐式重新聚焦视觉输入,并有效利用专家模型结果提升性能。我们在ReST和OmniDocBench上评估了DianJin-OCR-R1,结果表明其持续优于无推理版本及各类专家模型,验证了方法的有效性。

原文摘要 · Abstract (English)

Recent advances in vision-language models (VLMs) have enabled end-to-end document parsing and understanding, achieving strong performance on diverse optical character recognition (OCR) tasks. However, VLMs are prone to generate words that do not exist in the input image due to over-reliance on language priors. By contrast, traditional OCR models, whose architectures are tailored for specific recognition tasks, often achieve stronger fine-grained visual perception with fewer hallucinations, but they typically lack the contextual semantic understanding and reasoning capabilities needed in more challenging cases. To bridge this gap, we propose DianJin-OCR-R1, a reasoning-enhanced framework for recognition that trains VLMs in a reasoning-and-tool interleaved paradigm. Our DianJin-OCR-R1 model first recognizes the content in the input image through its own OCR capabilities, and then calls other expert models for extra results as references. After that, it is guided to "look again" at the image and compare its own recognized content with other results to find errors or omissions. Finally, it integrates all available evidence to generate a more accurate output. This design empowers the model to learn how to implicitly re-focus on the visual input and effectively leverage the results of other expert models for better performance. We evaluate our DianJin-OCR-R1 model on ReST and OmniDocBench, where it consistently outperforms both its non-reasoning counterparts and expert models, demonstrating the effectiveness of our method.

OCR视觉语言模型推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。