通过预判模型性能与成本，动态选择最优语言模型以平衡效率与精度。
Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning
- 基于检索的推理预测模型表现，不依赖固定模型列表。
- 可实现最高25.7%准确率提升或95.1%成本降低。
- 支持灵活调节精度与成本权衡，适配新模型与预算变化。
模型路由根据查询复杂度选择不同语言模型，将简单任务分配给低成本模型，复杂任务交由强模型，从而显著降低推理成本并保持高精度。然而，现有方法通常在有限模型集内进行静态选择，难以适应新模型或动态预算约束。本文提出SCOPE（可扩展且可控的成果性能评估器），不仅选择模型，还预测其成本与性能。通过强化学习训练，SCOPE利用检索相似问题中模型的表现进行推理，而非依赖固定模型名称，因此能兼容未见过的新模型。通过显式预测模型的准确率与开销，将路由转化为动态决策问题，用户可自由调控精度与成本的权衡。实验表明，当以性能为优先时，精度最高可提升25.7%；当注重效率时，成本最多可降低95.1%。
原文摘要 · Abstract (English)
Model routing chooses which language model to use for each query. By sending easy queries to cheaper models and hard queries to stronger ones, it can significantly reduce inference cost while maintaining high accuracy. However, most existing routers treat this as a fixed choice among a small set of models, which makes them hard to adapt to new models or changing budget constraints. In this paper, we propose SCOPE (Scalable and Controllable Outcome Performance Estimator), a routing framework that goes beyond model selection by predicting their cost and performance. Trained with reinforcement learning, SCOPE makes reasoning-based predictions by retrieving how models behave on similar problems, rather than relying on fixed model names, enabling it to work with new, unseen models. Moreover, by explicitly predicting how accurate and how expensive a model will be, it turns routing into a dynamic decision problem, allowing users to easily control the trade-off between accuracy and cost. Experiments show that SCOPE is more than just a cost-saving tool. It flexibly adapts to user needs: it can boost accuracy by up to 25.7% when performance is the priority, or cut costs by up to 95.1% when efficiency matters most.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。