动态构建多粒度证据图,精准定位长文档中的图文信息
MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA

- 用页面和元素构建可动态调整的证据图,融合布局与语义关系
- 在LongDocURL上达到52.75%准确率,MMLongBench-Doc上达53.26%
- 适合需要高精度跨模态长文档问答的科研与工业场景
长文档多模态问答需在大型PDF中定位稀疏证据,并整合文本、表格、图像、图表及复杂版式信息。现有RAG方法多依赖固定Top-k文本块或页面检索:文本检索压缩上下文但丢失视觉与版式信息;页面级视觉检索保留原始页面,却也引入大量无关区域,导致证据覆盖、噪声与推理成本间的静态权衡。本文提出MAGE-RAG,一种面向长文档多模态问答的多粒度自适应图证据框架。以页面检索为入口构建查询时证据图,离线构建包含页面节点与元素节点的证据图,编码包含关系、阅读顺序、版式邻接、章节层级与语义邻近等关系。查询时,在线证据控制器在明确预算下迭代激活、展开、搜索并剪枝证据。生成的证据子图被渲染为结构化多模态输入,使LVLM能在有限上下文内处理紧凑且相关的信息。在LongDocURL与MMLongBench-Doc上建立统一评估协议,涵盖直接MLLM、文本RAG、页面级视觉RAG及图/代理RAG。实验显示,MAGE-RAG在LongDocURL上达52.75%总体准确率,在MMLongBench-Doc上达53.26%准确率与51.19 F1。细粒度分析、预算-性能曲线、消融实验与追踪分析进一步表明,查询时证据子图构建能平衡分散证据覆盖与上下文噪声控制。
原文摘要 · Abstract (English)
Long-document multimodal question answering requires a system to locate sparse evidence in long PDFs and integrate clues from text, tables, images, charts, and complex layouts. Existing RAG methods mostly rely on fixed Top-k retrieval over text chunks or pages. Text retrieval can compress the context but often loses visual and layout information; page-level visual retrieval preserves the original page, yet it also sends large irrelevant regions to the reader, leading to a static trade-off among evidence coverage, noise, and inference cost. This paper proposes MAGE-RAG, a multigranular adaptive graph evidence framework for long-document multimodal QA. MAGE-RAG uses page retrieval as the entry point for query-time evidence construction. Offline, it builds an evidence graph with page nodes and element nodes, encoding containment, reading order, layout adjacency, section hierarchy, and semantic-neighbor relations. At query time, an online evidence controller iteratively activates, opens, searches, and prunes evidence under explicit budgets. The resulting evidence subgraph is then rendered into structured multimodal reader input, allowing the LVLM to consume compact and relevant evidence within a limited context. On LongDocURL and MMLongBench-Doc, we establish a unified comparison and analysis protocol covering Direct MLLM, Text RAG, Page-level Visual RAG, and Graph/Agentic RAG. Experiments show that MAGE-RAG achieves 52.75 overall accuracy on LongDocURL, and 53.26 accuracy with 51.19 F1 on MMLongBench-Doc. Fine-grained breakdowns, budget-performance curves, ablations, and trace-based analysis further show that query-time evidence subgraph construction can balance dispersed evidence coverage with context-noise control. Our code is available at https://github.com/laonuo2004/MAGE-RAG.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。