用子模函数选关键图，提升多图问答的准确率
Enhancing Multi-Image Question Answering via Submodular Subset Selection
- 用图切割等子模函数预选语义相关图像
- 在大规模图片库中显著提升检索效果
- 适合处理海量图像的问答系统开发者
大型多模态模型在单图视觉语言任务中表现优异，但在多图问答场景中表现不佳。此类任务涉及大量图像的推理，存在可扩展性差和检索性能下降的问题。本文针对MIRAGE模型中的检索框架，引入子模函数（如GraphCut）进行图像子集选择，通过查询感知机制预先筛选出语义相关的图像子集，再交由主检索模块处理。实验表明，采用基于锚点的查询与数据增强策略，能有效提升子模-检索管道在大规模图像集合中的表现。
原文摘要 · Abstract (English)
Large multimodal models (LMMs) have achieved high performance in vision-language tasks involving single image but they struggle when presented with a collection of multiple images (Multiple Image Question Answering scenario). These tasks, which involve reasoning over large number of images, present issues in scalability (with increasing number of images) and retrieval performance. In this work, we propose an enhancement for retriever framework introduced in MIRAGE model using submodular subset selection techniques. Our method leverages query-aware submodular functions, such as GraphCut, to pre-select a subset of semantically relevant images before main retrieval component. We demonstrate that using anchor-based queries and augmenting the data improves submodular-retriever pipeline effectiveness, particularly in large haystack sizes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。