通过推理增强实现检索性能提升,不靠模型变大。
Reason to Contrast: A Cascaded Multimodal Retrieval Framework
- 先推理再嵌入,用额外输入令牌提升检索能力。
- 在MMEB-V2上达75.7%准确率,2B模型媲美大型模型。
- 适合追求高效高精度检索的系统开发者。
传统多模态检索系统主要依赖双编码器架构,性能与嵌入维度紧密相关。近期工作Think-Then-Embed(TTE)表明,在嵌入前引入多模态推理以生成更多有意义的词元可进一步提升检索效果。本文提出TTE-v2,一种混合式多模态检索框架,通过增加输入词元预算而非扩大模型或嵌入尺寸,实现基于推理的性能扩展。该方法在初始检索后引入推理重排阶段,增强了查询与候选间的表达性交互,并为难负样本挖掘和假负样本过滤提供细粒度监督,形成反馈回路,有效强化上游检索器。这种级联设计实现了基于中间推理词元量的显著测试时改进。在MMEB-V2基准上的实验表明,TTE-v2-7B达到75.7%的新状态最优准确率,而TTE-v2-2B在使用更少外部数据的情况下,表现可匹敌甚至超越领先的7B模型。结果凸显了词元级扩展作为多模态检索替代扩展范式的价值。
原文摘要 · Abstract (English)
Traditional multimodal retrieval systems rely primarily on bi-encoder architectures, where performance is closely tied to embedding dimensionality. Recent work, Think-Then-Embed (TTE), shows that incorporating multimodal reasoning to elicit additional informative tokens before embedding can further improve retrieval. In this paper, we extend this paradigm with TTE-v2, a hybrid multimodal retrieval framework that introduces reasoning-driven performance scaling based on additional input token budget rather than model or embedding size. Our approach augments the initial multimodal retrieval with additional reasoning steps for reranking, enabling more expressive query-candidate interactions at test time. The reranking stage further provides fine-grained supervision for hard negative mining and false negative filtering, creating a feedback loop that effectively strengthens the upstream retriever. This cascaded design delivers substantial test-time improvements based on intermediate reasoning token scaling. Experiments on the MMEB-V2 benchmark demonstrate that TTE-v2-7B achieves a new state-of-the-art accuracy of 75.7%, and that TTE-v2-2B matches or surpasses leading 7B models trained with significantly larger external data. Our results highlight the promise of token-wise scaling as an alternative scaling paradigm for multimodal retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。