arXiv:2509.00520cs.IRcs.CL2025-09AAAI被引 8

用两阶段训练提升文本重排序效率与精度,兼顾速度与效果。

ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Reranking

  • 先用监督微调生成细粒度评分,增强相关性判别力。
  • 再用强化学习注入全局排序意识,提升排名准确性。
  • 在多个基准上表现领先,适合低延迟高精度场景。

文本重排序模型是现代检索增强生成系统的关键组件,用于在生成前筛选最相关的文档。当前基于大语言模型的重排序器常面临根本性权衡:基于监督微调的点式方法将相关性建模为二分类任务,缺乏足够评分区分度,尤其在推理型大模型上表现不足;而针对复杂推理的列表式方法虽性能强但计算效率低,难以满足低延迟应用需求。为此,我们提出ERank,一种基于推理型大模型的高效且有效的点式重排序器。采用新颖的两阶段训练流程:第一阶段通过监督微调,不再使用二值标签,而是让模型生成细粒度整数评分,显著提升相关性判别能力;第二阶段引入新型列表式导出奖励,通过强化学习进一步优化模型,使其具备全局排序感知。在BRIGHT、FollowIR、TREC DL和BEIR等基准上评估显示,ERank在有效性与鲁棒性上均优于现有方法。在强调推理能力的BRIGHT基准上,ERank-4B达到nDCG@10为38.7,更大版本32B达到40.2,创下新纪录。

原文摘要 · Abstract (English)

Text reranking models are a crucial component in modern systems like Retrieval-Augmented Generation, tasked with selecting the most relevant documents prior to generation. However, current Large Language Models (LLMs) powered rerankers often face a fundamental trade-off. On one hand, Supervised Fine-Tuning based pointwise methods that frame relevance as a binary classification task lack the necessary scoring discrimination, particularly for those built on reasoning LLMs. On the other hand, approaches designed for complex reasoning often employ powerful yet inefficient listwise formulations, rendering them impractical for low latency applications. To resolve this dilemma, we introduce ERank, a highly effective and efficient pointwise reranker built from a reasoning LLM that excels across diverse relevance scenarios. We propose a novel two-stage training pipeline that begins with Supervised Fine-Tuning (SFT). In this stage, we move beyond binary labels and train the model generatively to output fine grained integer scores, which significantly enhances relevance discrimination. The model is then further refined using Reinforcement Learning (RL) with a novel, listwise derived reward. This technique instills global ranking awareness into the efficient pointwise architecture. We evaluate the ERank reranker on the BRIGHT, FollowIR, TREC DL, and BEIR benchmarks, demonstrating superior effectiveness and robustness compared to existing approaches. On the reasoning-intensive BRIGHT benchmark, our ERank-4B achieves an nDCG@10 of 38.7, while a larger 32B variant reaches a state of the art nDCG@10 of 40.2.

文本重排序强化学习大模型高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。