MARVEL通过三步框架提升多模态检索，显著超越现有方法。
MARVEL: Multimodal Adaptive Reasoning-intensiVe Expand-rerank and retrievaL
- 用大模型扩展查询意图，增强语义覆盖
- 推理增强的密集检索器在MM-BRIGHT上达37.9 nDCG@10
- 适合需要深度理解的复杂多模态查询场景
在文本语料库上的多模态检索仍是核心挑战：当前最佳视觉语言编码器在MM-BRIGHT这一需要推理能力的多模态检索基准上仅取得27.6 nDCG@10的成绩，低于强文本系统表现。我们认为，有效的多模态检索需整合三项紧密协同的能力：扩展查询潜在意图、使用针对复杂推理训练的模型进行检索、基于显式分步推理对候选结果重排序。为此，我们提出 extbf{MARVEL}（多模态自适应推理增强型扩检重排与检索），一个统一流程，结合大模型驱动的查询扩展、专为复杂多模态查询优化的推理增强密集检索器（MARVEL-Retriever），以及基于GPT-4o的思维链重排与可选多轮互惠排序融合。在涵盖29个技术领域的MM-BRIGHT上评估，MARVEL达到37.9 nDCG@10，比最佳多模态编码器高出+10.3点，在27个领域超越所有单阶段基线，仅在两个高度专业领域（加密、量子计算）接近或匹配最优基线，证明推理密集型多模态检索应通过统一的扩检重排框架解决。
原文摘要 · Abstract (English)
Multimodal retrieval over text corpora remains a fundamental challenge: the best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, a reasoning-intensive multimodal retrieval benchmark, underperforming strong text-only systems. We argue that effective multimodal retrieval requires three tightly integrated capabilities that existing approaches address only in isolation: expanding the query's latent intent, retrieving with a model trained for complex reasoning, and reranking via explicit step-by-step reasoning over candidates. We introduce \textbf{MARVEL} (\textbf{M}ultimodal \textbf{A}daptive \textbf{R}easoning-intensi\textbf{V}e \textbf{E}xpand-rerank and retrieva\textbf{L}), a unified pipeline that combines LLM-driven query expansion, \textbf{MARVEL-Retriever} -- a reasoning-enhanced dense retriever fine-tuned for complex multimodal queries -- and GPT-4o-based chain-of-thought reranking with optional multi-pass reciprocal rank fusion. Evaluated on MM-BRIGHT across 29 technical domains, MARVEL achieves \textbf{37.9} nDCG@10, surpassing the best multimodal encoder by \textbf{+10.3 points} and outperforming all single-stage baselines in 27 of 29 domains and matching or approaching the best baseline in the remaining two highly-specialized domains (Crypto, Quantum Computing), demonstrating that reasoning-intensive multimodal retrieval is best addressed through a unified expand-retrieve-rerank framework. https://github.com/mm-bright/multimodal-reasoning-retrieval
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。