让大模型按需思考,提升排序效率与效果
Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking
- 用轻量路由头决定每条数据是否需要深度推理
- 在MovieLens上实现NDCG@10提升6.3%、令牌减少49.5%
- 适合资源受限场景下优化大模型排序性能
大型语言模型(LLMs)正被广泛应用于检索与推荐中的排序任务。尽管推理提示能提升排序效果,但我们的初步研究发现其收益不稳定且计算开销大,表明‘何时推理’与‘如何推理’同样关键。为此,我们提出一种推理路由框架,通过轻量级、即插即用的路由头,在生成前判断每个实例应采用直接推理(Non-Think)还是推理模式(Think)。该路由头仅依赖生成前信号:一是紧凑的排名感知特征(如候选分散度),二是基于诊断清单生成的模型感知难度信号,反映模型对推理的需求。利用这些信号,路由头输出一个可控标记,决定是否启用Think模式。此外,路由可在部署时沿验证帕累托前沿自适应选择策略,动态分配计算资源给最可能从推理中获益的样本。在三个不同规模的公开排名数据集上,使用多种开源LLM的实验表明,该方法在保持甚至提升排序性能的同时显著降低令牌消耗(例如在MovieLens上使用Qwen3-4B时,NDCG@10提升6.3%,令牌消耗减少49.5%),证明推理路由是解决精度-效率权衡的实际可行方案。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly applied to ranking tasks in retrieval and recommendation. Although reasoning prompting can enhance ranking utility, our preliminary exploration reveals that its benefits are inconsistent and come at a substantial computational cost, suggesting that when to reason is as crucial as how to reason. To address this issue, we propose a reasoning routing framework that employs a lightweight, plug-and-play router head to decide whether to use direct inference (Non-Think) or reasoning (Think) for each instance before generation. The router head relies solely on pre-generation signals: i) compact ranking-aware features (e.g., candidate dispersion) and ii) model-aware difficulty signals derived from a diagnostic checklist reflecting the model's estimated need for reasoning. By leveraging these features before generation, the router outputs a controllable token that determines whether to apply the Think mode. Furthermore, the router can adaptively select its operating policy along the validation Pareto frontier during deployment, enabling dynamic allocation of computational resources toward instances most likely to benefit from Think under varying system constraints. Experiments on three public ranking datasets with different scales of open-source LLMs show consistent improvements in ranking utility with reduced token consumption (e.g., +6.3\% NDCG@10 with -49.5\% tokens on MovieLens with Qwen3-4B), demonstrating reasoning routing as a practical solution to the accuracy-efficiency trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。