对比微调与对比学习,发现前者更适合大模型重排序。
Supervised Fine-Tuning or Contrastive Learning? Towards Better Multimodal LLM Reranking
- 将目标分解为权重和方向,揭示两类方法的差异机制。
- 微调在更新力度上显著强于对比学习,提升重排序性能。
- 适用于构建基于大模型的高效信息检索系统。
在信息检索中,重排序模型训练主要依赖两类目标:度量学习(如通过对比损失提升相关查询-文档对的预测得分)和分类(判断相关性与否的二分类)。对于BERT类编码器,研究显示对比学习(CL)优于判别式学习;但对大语言模型(LLM),监督微调(SFT)通过生成'是'/'否'标记来预测相关性,更契合其生成特性,表现更优。这种分歧引发核心问题:哪种目标内在更适合基于LLM的重排序?本文以通用多模态检索(UMR)为实验平台,将目标解耦为权重(控制更新幅度)与方向(指导更新方向),提出统一分析框架。探针实验表明,SFT在权重机制上显著优于CL,而评分方向无明显优劣。综合结果表明,SFT在LLM重排序中具有一致优势。我们通过大规模SFT训练,在MRB基准上取得新最优性能,并提供详尽的SFT设置消融分析,为该领域未来研究提供参考。
原文摘要 · Abstract (English)
In information retrieval, training reranking models mainly focuses on two types of objectives: metric learning (e.g. contrastive loss to increase the predicted scores on relevant query-document pairs) and classification (binary label prediction of relevance vs. irrelevance). For BERT-style encoders, various studies have shown that contrastive learning (CL) can be more effective than discriminative (classification) learning. However, for large language models (LLMs), classification via supervised fine-tuning (SFT), which predicts ''yes'' (resp. ''no'') token for relevant (resp. irrelevant) pairs, appears more promising as it aligns well with the generative nature of LLMs. This divergence raises a central question: which objective is intrinsically better suited to LLM-based reranking, and what mechanism underlies the difference? In this work, we conduct a comprehensive comparison and analysis between CL and SFT for reranking, taking the universal multimodal retrieval (UMR) as the experimental playground. We first decompose the objectives into two components: weight, which controls the magnitude of those updates, and direction, which guides the model updates, then present a unified framework for understanding their interactions. Through probing experiments, we find that SFT provides a substantially stronger weighting scheme than CL, whereas the preferred scoring direction shows no clear winner. Taken together, these results point to a consistent advantage of SFT over CL for LLM reranking. To further validate our findings, we conduct large-scale training with SFT and present new state-of-the-art rerankers on the MRB benchmark. We also provide ablations on SFT settings and expect our findings to benefit future research and applications in this area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。