arXiv:2605.14450cs.IR2026-05

让大模型少思考却更高效,实现低耗高质的文档重排序。

Stop Overthinking: Unlocking Efficient Listwise Reranking with Minimal Reasoning

  • 用自蒸馏方法提炼精炼推理路径,减少冗余思考。
  • 在TREC和NeuCLIR上降低34%-37%推理令牌消耗,性能不变。
  • 适合对响应速度敏感的检索系统部署场景。

基于大语言模型的列表级重排序已达到顶尖检索效果。近期,通过链式思维(CoT)进行深度对比分析的增强型模型进一步提升了性能,但代价是极高的计算开销——模型常生成数千个推理标记才得出最终排序。本文研究推理长度与排序质量的关系,发现存在‘过度思考’现象:延长推理反而收益递减。为此,我们提出长度正则化自蒸馏框架。通过从教师模型(Rank-K)中采样多样推理轨迹,并采用帕累托启发式筛选出以最少标记实现高排名性能的轨迹,构建合成数据集。学生模型在这些简洁、高质量的推理路径上微调,从而内化高效推理模式,有效剔除冗余推演。在TREC Deep Learning与NeuCLIR基准测试中,该方法在保持教师模型性能的同时,将推理令牌消耗降低34%-37%,为延迟敏感的应用提供了实用的推理增强重排序解决方案。

原文摘要 · Abstract (English)

Listwise reranking utilizing Large Language Models (LLMs) has achieved state-of-the-art retrieval effectiveness. Recently, reasoning-enhanced models have further pushed these boundaries by employing Chain-of-Thought (CoT) to perform deep comparative analysis of candidate documents. However, this performance gain comes at a prohibitive computational cost, as models often generate thousands of reasoning tokens before producing a final ranking. In this work, we investigate the relationship between reasoning length and ranking quality, revealing an overthinking phenomenon where extended reasoning yields diminishing returns. To address this, we propose a Length-Regularized Self-Distillation framework. We synthesize a dataset by sampling diverse reasoning traces from a teacher model (Rank-K) and applying a Pareto-inspired filter to select traces that achieve high ranking performance with minimal token usage. By fine-tuning on these concise, high-quality rationales, the student model learns to internalize efficient reasoning patterns, effectively pruning redundant deliberation. Experiments on TREC Deep Learning and NeuCLIR benchmarks demonstrate that our method maintains the teacher's effectiveness while reducing inference token consumption by 34%-37% across different retrieval settings, offering a practical solution for deploying reasoning-enhanced rerankers in latency-sensitive applications.

重排序大模型推理效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。