用模型自身置信度实现分步路由,省钱又提效。
Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning
- 基于小模型输出的置信度,动态决定是否调用大模型。
- 在AIME上提升20%准确率,同时减少48%计算量。
- 无需额外训练,适配多领域复杂推理任务。
大型语言模型(LLM)的推理能力虽因模型规模扩大和测试时技术进步而显著提升,但推理成本也随之增加。为降低开销,已有方法通过训练路由模型或延迟机制,将简单任务交由小型高效模型处理,复杂任务则交给大型昂贵模型。然而,这些外部训练的路由模块在领域变化下表现不稳定,且需借助蒙特卡洛滚动生成足够标注数据,成本高昂。本文提出无需外部模型的通用框架STEER,通过小模型在生成每一步推理前的置信度得分,实现细粒度、步骤级的模型调度。在数学推理、多跳问答与规划等多领域挑战性基准上的实验证明,STEER在保持甚至超越基线精度的同时,显著降低推理成本(如在AIME上实现+20%准确率,同时节省48% FLOPs),优于依赖外部模块的现有方法。结果表明,模型内部置信度是鲁棒且跨领域的有效路由信号,为高效部署大模型提供了可扩展路径。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) - particularly model scaling and test-time techniques - have greatly enhanced the reasoning capabilities of language models at the expense of higher inference costs. To lower inference costs, prior works train router models or deferral mechanisms that allocate easy queries to a small, efficient model, while forwarding harder queries to larger, more expensive models. However, these trained router models often lack robustness under domain shifts and require expensive data synthesis techniques such as Monte Carlo rollouts to obtain sufficient ground-truth routing labels for training. In this work, we propose Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (STEER), a domain-agnostic framework that performs fine-grained, step-level routing between smaller and larger LLMs without utilizing external models. STEER leverages confidence scores from the smaller model's logits prior to generating a reasoning step, so that the large model is invoked only when necessary. Extensive evaluations using different LLMs on a diverse set of challenging benchmarks across multiple domains such as Mathematical Reasoning, Multi-Hop QA, and Planning tasks indicate that STEER achieves competitive or enhanced accuracy while reducing inference costs (up to +20% accuracy with 48% less FLOPs compared to solely using the larger model on AIME), outperforming baselines that rely on trained external modules. Our results establish model-internal confidence as a robust, domain-agnostic signal for model routing, offering a scalable pathway for efficient LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。