让大模型路由更可靠，避免选错模型导致错误
RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models
- 提出新型路由方法RACER，可输出多个候选模型集
- 在多个测试集上显著提升下游任务准确率
- 适合对可靠性要求高的大模型系统部署
高效地将查询路由到最优的大语言模型（LLM）对于多模型系统中成本与性能的权衡至关重要。然而，现有大多数路由器依赖单一模型选择，容易发生误路由。本文将LLM路由建模为α-VOR问题，旨在最小化预期模型集合大小的同时控制误路由风险，提出新方法RACER，通过扩展基础路由器输出模型集合，并进行后续聚合以提升输出质量。RACER利用增强评分构建嵌套模型集合，并采用有限样本浓度界校准阈值，实现可变集合大小与拒答能力。理论证明，RACER在无需假设分布的前提下，以事后且模型无关的方式对未见测试数据实现了严格的无分布风险控制。大量实验验证了其理论保证，并表明RACER在多种基准测试中持续提升下游任务准确率。
原文摘要 · Abstract (English)
Efficiently routing queries to the optimal large language model (LLM) is crucial for optimizing the cost-performance trade-off in multi-model systems. However, most existing routers rely on single-model selection, making them susceptible to misrouting. In this work, we formulate LLM routing as the $α$-VOR problem to minimize expected set size while controlling the misrouting risk, and propose a novel method -- RACER, extending base routers to output model sets that can be subsequently aggregated for improved output. In particular, RACER constructs nested model sets via augmented scoring and utilizes finite-sample concentration bounds to calibrate a threshold that allows for both variable set sizes and abstention. We theoretically prove that RACER achieves rigorous distribution-free risk control on unseen test data in a post-hoc and model-agnostic manner. Extensive experiments verify our theoretical guarantees and demonstrate that RACER consistently enhances downstream accuracy across a wide range of benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。