用双线索检索+迭代精炼，轻量高效解决多页文档问答难题
SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinement
- 先按相似度粗选页面，再用摘要过滤重排，提升检索精准度
- 仅需较少页面检索即达3.2%性能提升，平均表现优于基线
- 适合需要高效处理多模态文档的工业级应用与研究者
文档视觉问答（DocVQA）是一项实用且具有挑战性的任务，需基于多页文档和多种模态信息（如图像、表格）回答问题。现有方法多采用检索增强生成（RAG）流程，利用视觉语言模型（VLM）嵌入并检索相关页面作为图像输入，再通过可接受图像的VLM生成答案。本文提出SimpleDoc，一种轻量但强大的检索增强框架。它通过双重线索策略：先基于嵌入相似性粗选候选页面，再依据页面摘要进行过滤与重排序，以提升证据页面获取效率。一个单一的VLM推理代理反复调用该双线索检索器，将新页面逐步引入工作记忆，直至问题被自信回答。在4个DocVQA数据集上，SimpleDoc平均性能超越先前基线3.2%，且检索页面数量显著减少。代码已开源：https://github.com/ag2ai/SimpleDoc。
原文摘要 · Abstract (English)
Document Visual Question Answering (DocVQA) is a practical yet challenging task, which is to ask questions based on documents while referring to multiple pages and different modalities of information, e.g, images and tables. To handle multi-modality, recent methods follow a similar Retrieval Augmented Generation (RAG) pipeline, but utilize Visual Language Models (VLMs) based embedding model to embed and retrieve relevant pages as images, and generate answers with VLMs that can accept an image as input. In this paper, we introduce SimpleDoc, a lightweight yet powerful retrieval - augmented framework for DocVQA. It boosts evidence page gathering by first retrieving candidates through embedding similarity and then filtering and re-ranking these candidates based on page summaries. A single VLM-based reasoner agent repeatedly invokes this dual-cue retriever, iteratively pulling fresh pages into a working memory until the question is confidently answered. SimpleDoc outperforms previous baselines by 3.2% on average on 4 DocVQA datasets with much fewer pages retrieved. Our code is available at https://github.com/ag2ai/SimpleDoc.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。