arXiv:2607.28580cs.AI2026-07中稿 · the 34th ACM Inter…

分离宏观推理与微观匹配,提升多模态问答的准确率

DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation

论文配图:DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation
图 1 · 摘自论文原文
  • 构建双图结构:宏观图负责全局推理,微观图专注局部证据匹配
  • 在多个数据集上,复杂问题回答准确率提升5.2%以上,证据召回率提高11.7%
  • 适合需要多跳推理的多模态问答任务,尤其对视觉细节敏感的应用

多模态检索增强生成(MM-RAG)在处理复杂多跳推理任务时仍面临挑战。现有方法多聚焦于实例级独立匹配,难以捕捉跨模态与文档间的显式关系。尽管图结构方法引入了结构建模,但在多模态场景中存在根本矛盾:细粒度视觉特征导致图快速膨胀并引入检索噪声,而粗粒度表示又会丢失关键局部证据。为此,我们提出DualG-MRAG,一种双层级框架,通过解耦宏观推理与微观匹配的图结构来应对该困境。具体而言,构建宏观图进行全局拓扑路由,微观图实现精确局部验证;通过图神经网络检索器设计查询驱动的消息传递过程,实现异构证据源间的动态相关性传播;进一步提出动态规划解码机制,从GNN前向传播中直接提取显式推理路径,替代传统孤立文档块输入。大量实验表明,DualG-MRAG在证据召回与复杂问答准确率上均优于基线方法。

原文摘要 · Abstract (English)

While Multimodal Retrieval-Augmented Generation (MM-RAG) has shown promising results, it still struggles with complex multi-hop reasoning tasks. Existing methods primarily focus on independent instance-level matching, which often fails to capture explicit relationships across modalities and documents. Although Graph-enhanced methods introduce structural modeling, they face a fundamental challenge in multimodal scenarios: incorporating fine-grained visual features leads to rapid graph expansion and retrieval noise, whereas coarse-grained representations cause the discarding of critical local evidence. To address this dilemma, we propose DualG-MRAG, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG. Specifically, to suppress retrieval noise by isolating global structural reasoning from fine-grained evidence matching, we construct a Macro Graph for global topological routing and a Micro Graph for precise local verification. Subsequently, to enable dynamic relevance propagation across heterogeneous evidence sources, we formulate retrieval as a query-driven message passing process via a GNN Retriever. Furthermore, to provide the generative model with coherent structural guidance, we introduce a dynamic programming decoding mechanism that extracts explicit reasoning paths directly from the GNN's forward pass, replacing the standard input of isolated document chunks. Extensive experiments demonstrate that DualG-MRAG outperforms baselines in both evidence recall and complex QA accuracy.

多模态推理增强图神经网络问答系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。