让AI检索图片细节而非整图，提升问答可验证性
From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG
- 将视觉元素作为独立检索单元，分三阶段精准定位目标
- 在真实地标数据集上提升29.2%准确率，超越6个基线模型
- 适合需要可解释性与高精度的多模态问答场景
多模态检索增强生成系统通常以整图或场景为单位检索证据，导致与细粒度用户问题不匹配，且错误难以验证。我们提出GranuVistaVQA，一个包含真实地标、多视角下元素级标注的多模态基准数据集，捕捉了单张图像仅呈现部分实体的局部观测挑战。进一步提出GranuRAG，一种多粒度框架，通过三个阶段实现：元素级检测与分类、多粒度跨模态对齐检索证据、以及归因约束生成。该方法将检索基础置于元素层面，而非依赖隐式注意力机制，实现错误透明诊断。实验表明，GranuRAG在该任务上相较六个强基线模型最高提升29.2%。
原文摘要 · Abstract (English)
Multimodal Retrieval-Augmented Generation (RAG) systems retrieve evidence at coarse granularities (entire images or scenes), creating a mismatch with fine-grained user queries and making failures unverifiable. We introduce GranuVistaVQA, a multimodal benchmark featuring real-world landmarks with element-level annotations across multiple viewpoints, capturing the partial observation challenge where individual images contain only subsets of entities. We further propose GranuRAG, a multi-granularity framework that treats visual elements as first-class retrieval units through three stages: element-level detection and classification, multi-granularity cross-modal alignment for evidence retrieval, and attribution-constrained generation. By grounding retrieval at the element level rather than relying on implicit attention, our approach enables transparent error diagnosis. Experiments demonstrate that GranuRAG achieves up to 29.2% improvement over six strong baselines for this task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。