arXiv:2602.21053cs.CV2026-02被引 1

让OCR模型会自我反思,纠错更准更快

OCR-Agent: Agentic OCR with Capability and Memory Reflection

  • 通过能力与记忆双重反思机制诊断错误并规划修正
  • 在OCRBench v2上中文和英文准确率分别提升1.2和2.0点
  • 无需额外训练,适合需要高可靠性的视觉理解场景

大型视觉语言模型在复杂视觉理解任务中表现出巨大潜力,但普遍缺乏有效的自我修正机制,难以独立纠正认知偏差。多轮修订中常陷入重复无效尝试,无法稳定提升答案质量。为此,我们提出一种新型迭代自修正框架,赋予模型两项核心能力:能力反思与记忆反思。该框架引导模型先通过能力反思诊断错误并生成修正计划,再利用记忆反思回顾过往尝试以避免重复、探索新解法,最后通过严格重推理优化答案。在挑战性OCRBench v2基准测试中,OCR-Agent在英文子集上超越当前开源最优模型InternVL3-8B 2.0分,在中文子集上提升1.2分,视觉理解与推理得分分别达79.9和66.5,超过更大微调模型。结果表明,结构化、自知的反思能显著提升视觉语言模型的推理鲁棒性,且无需额外训练。

原文摘要 · Abstract (English)

Large Vision-Language Models (VLMs) have demonstrated significant potential on complex visual understanding tasks through iterative optimization methods.However, these models generally lack effective self-correction mechanisms, making it difficult for them to independently rectify cognitive biases. Consequently, during multi-turn revisions, they often fall into repetitive and ineffective attempts, failing to achieve stable improvements in answer quality.To address this issue, we propose a novel iterative self-correction framework that endows models with two key capabilities: Capability Reflection and Memory Reflection. This framework guides the model to first diagnose errors and generate a correction plan via Capability Reflection, then leverage Memory Reflection to review past attempts to avoid repetition and explore new solutions, and finally, optimize the answer through rigorous re-reasoning. Experiments on the challenging OCRBench v2 benchmark show that OCR-Agent outperforms the current open-source SOTA model InternVL3-8B by +2.0 on English and +1.2 on Chinese subsets, while achieving state-of-the-art results in Visual Understanding (79.9) and Reasoning (66.5) - surpassing even larger fine-tuned models. Our method demonstrates that structured, self-aware reflection can significantly enhance VLMs' reasoning robustness without additional training. Code: https://github.com/AIGeeksGroup/OCR-Agent.

OCR自修正视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。