用检索增强生成提升文档视觉问答准确率
Enhancing Document VQA Models via Retrieval-Augmented Generation
- 先检索相关段落再生成答案，减少内存占用
- 文本检索法比全页拼接提升22.5分ANLS
- 无需文字提取的纯视觉检索也有效
文档视觉问答需处理数十页文档，现有系统多采用拼接所有页面或依赖超大模型，均消耗大量内存。检索增强生成（RAG）提供新思路：先检索相关段落，再基于选中证据生成答案。本文系统评估了不同检索方式在多模型、多基准上的效果，包括基于OCR文本和无OCR的纯视觉检索。在多页数据集MP-DocVQA、DUDE和InfographicVQA上，文本主导的RAG方法相比全页拼接基线最高提升22.5 ANLS；纯视觉检索在无需文本提取的情况下仍实现+5.0 ANLS提升。消融实验证明，检索与重排序组件贡献主要增益，而近期提出的布局引导分块策略在这些数据集上未带来帮助。实验表明，精心选择证据能持续提升多种规模模型与多页基准的准确率，具实际应用价值。
原文摘要 · Abstract (English)
Document Visual Question Answering (Document VQA) must cope with documents that span dozens of pages, yet leading systems still concatenate every page or rely on very large vision-language models, both of which are memory-hungry. Retrieval-Augmented Generation (RAG) offers an attractive alternative, first retrieving a concise set of relevant segments before generating answers from this selected evidence. In this paper, we systematically evaluate the impact of incorporating RAG into Document VQA through different retrieval variants - text-based retrieval using OCR tokens and purely visual retrieval without OCR - across multiple models and benchmarks. Evaluated on the multi-page datasets MP-DocVQA, DUDE, and InfographicVQA, the text-centric variant improves the "concatenate-all-pages" baseline by up to +22.5 ANLS, while the visual variant achieves +5.0 ANLS improvement without requiring any text extraction. An ablation confirms that retrieval and reranking components drive most of the gain, whereas the layout-guided chunking strategy - proposed in several recent works to leverage page structure - fails to help on these datasets. Our experiments demonstrate that careful evidence selection consistently boosts accuracy across multiple model sizes and multi-page benchmarks, underscoring its practical value for real-world Document VQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。