arXiv:2607.24748cs.IRcs.AI2026-07

让AI读懂图文混排长文档,自动找证据并验证答案可靠性。

VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents

论文配图:VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents
图 1 · 摘自论文原文
  • 用图文混合索引+关键词与语义双检索,精准定位相关页面。
  • 在长文档问答任务中,证据页召回率提升,答案准确率显著更高。
  • 适合需要跨页推理的报告、手册类复杂文档理解场景。

图文混排的长文档(如报告、幻灯片、说明书)常将问题答案分散在多页中,融合文本、版式、表格、图表等多模态信息。本文研究面向此类文档的多模态检索增强生成,要求检索能同时捕获文本和视觉信号。提出VLD-RAG框架,构建保留页面结构的多模态索引,存储解析文本、页面元数据及密集视觉表示,并采用关键词稀疏搜索与语义稠密查询相结合的混合检索策略,识别候选来源与证据页。通过验证器引导的代理工作流,协调检索代理、回答代理与验证代理,扩大证据覆盖范围,检测缺失引用,并动态优化检索请求。在LongDocURL和MMLongBench-Doc等基准上,以Top-1和Top-5证据页准确率评估检索,以广义准确率评估生成,结果表明VLD-RAG优于现有基于视觉的检索基线,在证据页检索与端到端问答任务中均有提升。说明协同代理验证与多模态混合检索对依赖跨页证据的可靠答案生成至关重要。

原文摘要 · Abstract (English)

Visually-rich documents such as reports, slides, and manuals often distribute the evidence needed to answer a question across multiple pages, mixing text with layout cues, tables, charts, and figures. This work studies multimodal retrieval-augmented generation for question answering over such visually-rich long documents, where retrieval must select evidence pages that include both textual and visual signals. We present VLD-RAG, an agentic multimodal RAG framework for multi-page evidence retrieval and cross-page reasoning over long documents. VLD-RAG builds a page-preserving multimodal index that stores parsed text, page-level metadata, and dense visual representations, and uses a hybrid retrieval strategy that combines keyword-based sparse search with dense semantic queries to identify candidate sources and evidence pages. A verifier-guided agent workflow coordinates a Retrieval Agent, Answer Agent, and Validation Agent to broaden evidence coverage, detect missing citations, and refine retrieval requests when needed. We evaluate retrieval with Top-1 and Top-5 evidence-page accuracy and generation with generalized accuracy, and show that VLD-RAG improves both evidence-page retrieval and end-task question answering on visually-rich long-document benchmarks, including LongDocURL and MMLongBench-Doc, outperforming previous vision-based retrieval baselines. These findings highlight that coordinated agent verification and multimodal hybrid retrieval are crucial for reliable grounding when correct answers depend on evidence scattered across pages.

多模态长文档RAG智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。