用多向量匹配提升图文关联检索，显著改善视觉证据的利用率。
ColGraphRAG: Late-Interaction Evidence Retrieval for Multimodal GraphRAG
- 采用晚交互多向量打分替代单向量相似度，保留图像细粒度结构
- 在MultimodalQA上图像候选召回率提升，问答准确率显著增长
- 适合需要精细视觉推理的任务，尤其图像信息关键的场景
基于图结构的多模态问答将文本、表格和图像组织为证据图，但端到端精度依赖于哪些多模态内容能被充分排序进入下游推理。对于图关联的图像，单一向量双编码器相似度会丢失补丁级和标记级结构，影响细粒度对齐。本文在ColBERT/ColPali框架下，将图关联图像节点的候选排序操作替换为晚交互式多向量评分（MaxSim风格），同时保持离线图构建、文本与表格检索、结构化提取及下游推理不变。在MultimodalQA数据集上，该改进带来图关联图像候选在检索阶段点估计的提升，并推动下游问答性能增益，在视觉证据主导的任务中提升更明显，而文本主导任务表现混合；这一趋势为图关联视觉证据的有效性提供了机制层面的证据，但更广泛的验证与更精细的图级诊断仍是未来工作重点。
原文摘要 · Abstract (English)
Graph-grounded multimodal question answering organizes text, tables, and images in a structured evidence graph, yet end-to-end accuracy depends on which multimodal assets are ranked highly enough to enter downstream reasoning; for graph-linked images, single-vector bi-encoder similarity can discard patch- and token-level structure needed for fine-grained alignment. We evaluate replacing the visual candidate-ranking operator over graph-linked image nodes with late-interaction MaxSim-style multi-vector scoring in the ColBERT/ColPali lineage, while keeping offline graph construction, text- and table-side retrieval, structured extraction, and downstream reasoning unchanged. On MultimodalQA, this change is associated with improved retrieval-stage point estimates for graph-linked image candidates and downstream QA gains, with larger movement where visual evidence matters most and mixed trends on text-dominant questions; we interpret the pattern as mechanism-level evidence for graph-linked visual evidence inclusion, while broader validation and finer graph-level diagnostics remain important future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。