arXiv:2604.16313cs.IRcs.AI2026-04被引 3

提出自适应多模态检索增强框架,提升复杂文档问答准确性

MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering

论文配图:MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering
图 1 · 摘自论文原文
  • 根据问题动态调整文档表示和检索重点
  • 在6个基准上超越当前最优方法,提升检索相关性和答案质量
  • 适合需要精准理解图文混合文档的场景

基于检索的多模态文档问答旨在从视觉丰富的复杂结构文档中识别并整合相关信息。尽管检索增强生成(RAG)在文本问答中表现优异,但其向多模态文档的扩展仍不充分,面临显著挑战。现有方法依赖与查询无关的文档表示,忽略关键内容,并采用静态top-k证据选择,无法适应相关信息分布的不确定性。为此,我们提出多模态自适应检索增强(MARA)框架,引入查询自适应机制于检索与生成阶段。MARA包含两个组件:查询对齐区域编码器,构建多层次文档表示并依据查询相关性重加权,提升检索精度;自省式证据控制器,在生成过程中监测证据充分性,采用滑动窗口策略动态引入低排名内容。在六个多模态QA基准上的实验表明,MARA在检索相关性和答案质量上持续优于现有SOTA方法。

原文摘要 · Abstract (English)

Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures. While retrieval-augmented generation (RAG) has shown strong performance in text-based QA, its extensions to multimodal documents remain underexplored and face significant limitations. Specifically, current approaches rely on query-agnostic document representations that overlook salient content and use static top-k evidence selection, which fails to adapt to the uncertain distribution of relevant information. To address these limitations, we propose the Multimodal Adaptive Retrieval-Augmented (MARA) framework, which introduces query-adaptive mechanisms to both retrieval and generation. MARA consists of two components: a Query-Aligned Region Encoder that builds multi-level document representations and reweights them based on query relevance to improve retrieval precision; and a Self-Reflective Evidence Controller that monitors evidence sufficiency during generation and adaptively incorporates content from lower-ranked sources using a sliding-window strategy. Experiments on six multimodal QA benchmarks demonstrate that MARA consistently improves retrieval relevance and answer quality over existing SOTA method.

多模态问答检索增强自适应机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。