让AI读懂长文档,精准定位关键证据并推理答案
DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding

- 采用分析-定位-推理的结构化流程,提升长文档理解能力
- 在多页文档上实现高精度答案与证据定位,支持超长文档泛化
- 适合需要精准信息抽取的长文档问答系统开发者使用
现有多模态大模型在长文档理解任务中随文档长度增加性能显著下降,主要源于两个根本挑战:1)信号噪声比低,关键证据被无关页面淹没;2)监督信号稀缺,现有数据集仅提供简短答案,学习信号微弱。本文提出一种新范式,要求模型执行结构化的分析、定位与推理流程。为训练该能力,设计两阶段训练框架:首先通过高效知识蒸馏生成高质量数据进行监督微调;随后采用面向证据的组相对策略优化,联合优化证据定位与答案准确性。此外,引入证据引导的资源分配策略,缓解多页文档训练中的内存压力。大量实验表明,DocSeeker在域内与域外任务上均表现优异,能从短页训练有效泛化至超长文档,并天然适配视觉增强生成系统,可作为其坚实基础。
原文摘要 · Abstract (English)
Existing Multimodal Large Language Models (MLLMs) suffer from significant performance degradation on the long document understanding task as document length increases. This stems from two fundamental challenges: 1) a low Signal-to-Noise Ratio (SNR), with crucial evidence buried in irrelevant pages; and 2) supervision scarcity, as datasets offering only final short answers provide a weak learning signal. In this paper, we address these challenges by proposing a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow. To instill this capability, we design a two-stage training framework: we first perform Supervised Fine-Tuning on high-quality data generated via an efficient knowledge distillation strategy. Subsequently, we employ an Evidence-aware Group Relative Policy Optimization which jointly optimizes for both evidence localization and answer accuracy. Additionally, we introduce a Evidence-Guided Resolution Allocation strategy to mitigate memory constraints of training on multi-pages documents. Extensive experiments demonstrate that DocSeeker achieves superior performance on both in-domain and out-of-domain tasks. We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems, serving as a solid foundation for their implementation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。