arXiv:2607.16604eess.IV2026-07

验证图结构与多模态增强在文档问答中的真实效果,发现只有改变检索才有效。

Controlled Evaluation of Graph and Multimodal Augmentation in RAG for Document Question Answering

  • 用控制实验分离检索、上下文、视觉等影响因素
  • 图增强在证据不足时提升检索完整度,准确率从0.22升至0.46
  • 多模态增益易受文本泄露干扰,需验证答案无法仅靠文本获取

图结构与多模态扩展常在端到端评估中混杂多种因素,难以判断性能提升源自何处。本文通过五种RAG配置、四种多模态生成器和三个文档语料库,进行分阶段且证据可控的评估。使用相同大模型提取的知识图谱,分别作为检索后三元组注入(+KG)或检索中实体桥接的段落扩展(+KGret)。提示侧注入图结构未带来一致准确率提升,反而降低忠实性;而+KGret使HotpotQA桥接问题的黄金证据完整度从0.22提升至0.46,SPIQA跨论文问题从0.50升至0.72,在所有生成器上均提升准确率,对完整证据对照组影响甚微。多模态评估中,匹配标题-可答与像素仅有的协议显示,看似显著的多模态增益高度依赖文本泄漏。程序化检查表明答案可从标题、语料文本或模型响应中恢复,无需完整黄金证据。在广泛传播语料上,不完整证据问题准确率达0.35–0.71,而PubLayNet上为0,说明原始准确率可能夸大了检索贡献。结果表明,图增强最有效的场景是缓解证据缺失下的检索缺陷,而多模态评估必须明确验证答案无法仅通过文本获得。

原文摘要 · Abstract (English)

Graph and multimodal extensions to retrieval-augmented generation (RAG) are often evaluated end to end, making it difficult to isolate whether gains arise from retrieval, prompt-side context, visual access, generator capability, or benchmark construction. We present a stage- and evidence-controlled evaluation across five RAG configurations, four multimodal generators, and three document corpora. The same LLM-extracted knowledge graph is used either after retrieval as provenance-constrained triple injection (+KG) or during retrieval as entity-bridged passage expansion (+KGret). Prompt-side graph injection yields no consistent accuracy improvement and generally reduces faithfulness. In contrast, +KGret increases gold-evidence completeness from 0.22 to 0.46 on HotpotQA bridge questions and from 0.50 to 0.72 on SPIQA cross-paper questions, improving accuracy for every generator on both evidence-deficient sets while having little effect on retrieval-complete controls. For visual question answering, matched caption-answerable and verified pixel-only protocols show that apparent multimodal gains are sensitive to textual leakage. Programmatic checks reveal answer recoverability from captions, corpus text, and model responses generated without complete gold evidence. Accuracy on incomplete-evidence questions reaches 0.35--0.71 on widely disseminated corpora, compared with 0 on PubLayNet, indicating that raw accuracy can overstate retrieval-attributable performance. These results show that graph augmentation is most effective when it changes retrieval under evidence deficits, while multimodal evaluation requires explicit verification that answers are unavailable through text.

RAG图增强多模态评估证据完整性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。