arXiv:2608.15056cs.AI2026-08

GraphLoom通过可信度校准路由精炼证据,提升多模态知识图谱检索生成的准确与可靠。

GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG

论文配图:GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG
图 1 · 摘自论文原文
  • 构建实例级多模态知识图谱,按可信度筛选并分层路由高价值证据。
  • 在多个数据集上显著提升答案质量与证据忠实性,噪声环境下表现稳定。
  • 适合需要高可信推理的多模态问答场景,尤其应对复杂干扰信息时优势明显。

多模态检索增强生成(RAG)系统常依赖长而无结构的上下文或过度扩展的证据图,导致噪声引入、多跳推理能力下降及生成不可靠。本文提出GraphLoom,一种可靠性校准的多模态知识图谱RAG框架,实现紧凑且忠实的证据路由。给定问题和关联的多模态输入,GraphLoom从场景描述、抽取的关系三元组和外部常识知识中构建实例级多模态知识图谱。不将所有检索证据注入生成器,而是通过受限扩展的可信度感知子图检索,利用分层图内存槽与联合图-序列注意力,在冻结的语言模型中选择性地路由高实用性证据。为增强复杂推理下的鲁棒性,GraphLoom进一步结合交错检索与预算修正检索,实现在噪声检索条件下的自适应多跳证据优化。我们在ScienceQA、MultiModalQA和OK-VQA上进行评估,包含模拟外部知识检索噪声的大干扰证据池。实验结果表明,相比强基线的多模态RAG、图检索及开源视觉-语言模型,GraphLoom在答案质量和证据忠实性上持续提升,在MultiModalQA上检索质量改善,且在噪声证据池下性能稳定。额外分析使用MiniCheck验证、人工评估和延迟分析显示,可靠性校准的图证据路由是长上下文多模态证据注入的有效替代方案。

原文摘要 · Abstract (English)

Multimodal retrieval-augmented generation (RAG) systems often rely on long unstructured contexts or aggressively expanded evidence graphs, which can introduce noisy evidence, weaken multi-hop reasoning, and increase unsupported generation. We present GraphLoom, a reliability-calibrated multimodal knowledge-graph RAG framework for compact and faithful evidence routing. Given a question and its associated multimodal input, GraphLoom constructs an instance-level multimodal knowledge graph from grounded scene descriptions, extracted relational triples, and external commonsense knowledge. Instead of injecting all retrieved evidence into the generator, GraphLoom performs reliability-aware subgraph retrieval with bounded expansion and selectively routes high-utility evidence through hierarchical graph memory slots and joint graph-sequence attention in a frozen language model. To improve robustness in complex reasoning settings, GraphLoom further combines interleaved retrieval with budgeted corrective retrieval, enabling adaptive multi-hop evidence refinement under noisy retrieval conditions. We evaluate GraphLoom on ScienceQA, MultiModalQA, and OK-VQA, including large distractor evidence pools that approximate noisy external knowledge retrieval. Experimental results show consistent gains in answer quality and evidence faithfulness over strong multimodal RAG, graph-retrieval, and open-source vision-language baselines, with improved retrieval quality on MultiModalQA and stable performance under noisy evidence pools. Additional analyses using MiniCheck-based verification, human evaluation, and latency profiling show that reliability-calibrated graph evidence routing provides an effective alternative to long-context multimodal evidence injection.

多模态知识图谱RAG可信度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。