通过强化学习优化多模态查询，提升图像转文本检索效果。
BRIDGE: Multimodal-to-Text Retrieval via Reinforcement-Learned Query Alignment
- 用强化学习将杂乱的多模态查询提炼为简洁检索词。
- 在MM-BRIGHT上达29.7 nDCG@10，超越所有带多模态编码器的基线。
- 可插拔适配现有模型，适合需要精准检索的多模态应用。
多模态检索系统在面对纯文本语料库时表现不佳：最佳视觉语言编码器在MM-BRIGHT数据集上仅取得27.6 nDCG@10，落后于强文本检索器。我们提出瓶颈不在检索器，而在查询——原始多模态查询混杂了视觉描述、对话噪声和检索意图，严重降低嵌入相似性。本文提出BRIDGE系统，由两部分组成：FORGE（聚焦检索查询生成器）通过强化学习将噪声查询浓缩为紧凑、优化的检索字符串；LENS（语言增强型神经检索器）基于推理密集型数据微调，处理FORGE生成的高意图查询。在包含2,803个查询、29个领域的MM-BRIGHT上评估，BRIDGE达到29.7 nDCG@10，超越所有含多模态编码器的基线（如Nomic-Vision，27.6）。当FORGE作为插件用于Nomic-Vision时，系统达33.3 nDCG@10，超过最佳纯文本检索器（32.2），证明查询对齐是多模态转文本检索的核心瓶颈。
原文摘要 · Abstract (English)
Multimodal retrieval systems struggle to resolve image-text queries against text-only corpora: the best vision-language encoder achieves only 27.6 nDCG@10 on MM-BRIGHT, underperforming strong text-only retrievers. We argue the bottleneck is not the retriever but the query -- raw multimodal queries entangle visual descriptions, conversational noise, and retrieval intent in ways that systematically degrade embedding similarity. We present \textbf{BRIDGE}, a two-component system that resolves this mismatch without multimodal encoders. \textbf{FORGE} (\textbf{F}ocused Retrieval Query Generato\textbf{r}) is a query alignment model trained via reinforcement learning, which distills noisy multimodal queries into compact, retrieval-optimized search strings. \textbf{LENS} (\textbf{L}anguage-\textbf{E}nhanced \textbf{N}eural \textbf{S}earch) is a reasoning-enhanced dense retriever fine-tuned on reasoning-intensive retrieval data to handle the intent-rich queries FORGE produces. Evaluated on MM-BRIGHT (2,803 queries, 29 domains), BRIDGE achieves \textbf{29.7} nDCG@10, surpassing all multimodal encoder baselines including Nomic-Vision (27.6). When FORGE is applied as a plug-and-play aligner on top of Nomic-Vision, the combined system reaches \textbf{33.3} nDCG@10 -- exceeding the best text-only retriever (32.2) -- demonstrating that \textit{query alignment} is the key bottleneck in multimodal-to-text retrieval. https://github.com/mm-bright/multimodal-reasoning-retrieval
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。