arXiv:2503.23427cs.CLcs.IR2025-03EMNLP被引 16

小模型预排+大模型精排,效率提升70%且效果更好

CoRanking: Collaborative Ranking with Small and Large Ranking Agents

  • 用小模型先筛选候选文档,再让大模型只重排前20个
  • 相比纯大模型方法,延迟降低约70%,效果反而更优
  • 通过强化学习调整排序顺序,缓解大模型位置偏见

大型语言模型(LLM)在列表级排序任务中表现优异,但其性能通常依赖于大规模参数(如GPT-4)和重复滑动窗口过程,带来显著的效率挑战。本文提出一种名为CoRanking的协同排序框架,将小型与大型排序模型结合,实现高效且有效的排序。CoRanking首先使用小型重排序器对所有候选段落进行预排序,将相关段落前置至列表前部(如前20位)。随后,仅对这些高置信度段落应用LLM列表级重排序器,大幅提高整体效率。然而,已有研究指出LLM列表级重排序器存在明显的输入段落顺序偏好。若直接输入小模型选出的前20个段落,可能因顺序不匹配导致大模型性能下降。为此,我们引入一个基于强化学习训练的段落顺序调整器,对小模型输出的前段落进行重排序,使其更符合大模型的偏好。在三个信息检索基准上的大量实验表明,CoRanking在显著提升效率(排名延迟减少约70%)的同时,效果优于仅使用大模型的方法。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated superior listwise ranking performance. However, their superior performance often relies on large-scale parameters (\eg, GPT-4) and a repetitive sliding window process, which introduces significant efficiency challenges. In this paper, we propose \textbf{CoRanking}, a novel collaborative ranking framework that combines small and large ranking models for efficient and effective ranking. CoRanking first employs a small-size reranker to pre-rank all the candidate passages, bringing relevant ones to the top part of the list (\eg, top-20). Then, the LLM listwise reranker is applied to only rerank these top-ranked passages instead of the whole list, substantially enhancing overall ranking efficiency. Although more efficient, previous studies have revealed that the LLM listwise reranker have significant positional biases on the order of input passages. Directly feed the top-ranked passages from small reranker may result in the sub-optimal performance of LLM listwise reranker. To alleviate this problem, we introduce a passage order adjuster trained via reinforcement learning, which reorders the top passages from the small reranker to align with the LLM's preferences of passage order. Extensive experiments on three IR benchmarks demonstrate that CoRanking significantly improves efficiency (reducing ranking latency by about 70\%) while achieving even better effectiveness compared to using only the LLM listwise reranker.

排序LLM效率优化协同机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。