让文档理解模型像人一样看图推理,证据与布局紧密结合。
DocCogito: Aligning Layout Cognition and Step-Level Grounded Reasoning for Document Understanding
- 用结构化视觉-语义链替代自由文本推理,精准对齐图文区域。
- 在6个基准上4项刷新纪录,关键任务提升超过5%准确率。
- 适合需要高可信度推理的医疗、法律等专业文档场景。
多模态大模型进行文档理解时,不仅需要准确答案,还需可解释、有证据支撑的推理过程,尤其在高风险场景中更为重要。然而,现有模型虽改进了版面编码和思维链式提示,但两者之间的交互通常为隐式学习,耦合松散,缺乏系统性机制。为此,我们提出DocCogito,一个统一框架,将全局版面感知与结构化、区域对齐的推理相结合。该框架引入轻量级版面塔,将页面结构提炼为可学习的全局版面先验标记;并设计确定性的视觉-语义链(VSC),以紧凑且无歧义的结构化表示,监督细粒度中间推理过程,使其与证据区域对齐。训练采用渐进式策略:版面感知预训练、基于VSC的冷启动、拒绝采样和GRPO优化。为进一步强化版面先验与VSC执行间的内在关联,我们在标准奖励基础上增加细粒度区域置信信号,促使推理轨迹始终与对应证据区域保持一致。在六个基准(DocVQA、WTQ、ChartQA、TextVQA、OCRBench、InfoVQA)上的大量实验表明,该方法具备强泛化能力,在四个基准上达到当前最优表现。
原文摘要 · Abstract (English)
Document understanding with multimodal large language models (MLLMs) requires not only accurate answers but also explicit, evidence-grounded reasoning, especially in high-stakes scenarios. However, current document MLLMs still fall short of forming a complete, human-like reasoning process, because even when they improve both layout encoding and CoT-style prompting, the interaction between the two is typically learned implicitly and remains loosely coupled rather than being enforced as a systematic mechanism. So we propose DocCogito, a unified framework that integrates global layout perception with structured, region-grounded reasoning. DocCogito introduces a lightweight layout tower that distills page structure into learnable global layout prior tokens, and a deterministic Visual-Semantic Chain (VSC)-a concise structured representation less ambiguous than free-form natural-language CoT-to supervise fine-grained intermediate reasoning aligned with evidence regions. Training follows a progressive recipe, including layout perception pretraining, VSC-guided cold start, rejection sampling, and GRPO. To further strengthen the internal coupling between layout priors and VSC execution, we augment standard rewards with a fine-grained region-confidence signal that encourages reasoning traces to stay aligned with corresponding evidence regions. Extensive experiments on six benchmarks (DocVQA, WTQ, ChartQA, TextVQA, OCRBench, and InfoVQA) demonstrate strong generalization, achieving state-of-the-art results on four benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。