让大模型在排序任务中表现更好,同时不丢通用能力。
RaCT: Ranking-aware Chain-of-Thought Optimization for LLMs
- 用思维链提示+两阶段训练,让模型推理过程更透明。
- 在TREC数据集上优于RankZephyr,nDCG等指标全面提升。
- 兼顾专业排序与通用推理,适合需要双能力的场景。
在信息检索中,大语言模型凭借其强大的自然语言理解与推理能力,在文本重排任务中展现出巨大潜力。然而,传统监督微调方法在提升模型排序性能的同时,常导致其通用能力显著退化。为此,本文提出一种新方法:将思维链(Chain-of-Thought)提示与两阶段训练流程(监督微调后接排序偏好优化,SFT-RPO)相结合。思维链促使模型在排序决策中显式表达推理过程,形成从查询-文档分析到最终评分的透明路径,同时保持微调过程中分析能力的完整性。在TREC深度学习数据集上的大量实验表明,该方法性能优于现有最优模型如RankZephyr,多个评估指标(如nDCG)均持续提升。更重要的是,在大规模多任务语言理解(MMLU)基准上的全面评估显示,该方法在多样化推理任务中仍保持稳健表现,为通过策略性微调实现排序性能提升与通用能力保留提供了有力实证。
原文摘要 · Abstract (English)
In information retrieval, large language models (LLMs) have demonstrated remarkable potential in text reranking tasks by leveraging their sophisticated natural language understanding and advanced reasoning capabilities. However, conventional supervised fine-tuning approaches for specializing LLMs in ranking tasks often lead to significant degradation of the models' general-purpose abilities. To address this fundamental challenge, this paper presents a novel methodology that strategically combines Chain-of-Thought (CoT) prompting techniques with an innovative two-stage training pipeline consisting of Supervised Fine-Tuning followed by Ranking Preference Optimization (SFT-RPO). The Chain-of-Thought prompting component encourages models to explicitly articulate their reasoning process during ranking decisions, creating a transparent pathway from query-document analysis to final ranking scores while maintaining analytical capabilities throughout fine-tuning. Extensive experimental evaluations on the TREC Deep Learning datasets demonstrate that our proposed method achieves superior performance compared to existing state-of-the-art models, including RankZephyr, showing consistent improvements across multiple evaluation metrics such as normalized Discounted Cumulative Gain (nDCG). Most significantly, comprehensive assessments on the Massive Multitask Language Understanding (MMLU) benchmark reveal that our method successfully maintains robust performance across diverse reasoning tasks, providing strong empirical evidence for effective retention of general-purpose capabilities through strategic fine-tuning while achieving specialized performance improvements in text reranking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。