用置信度指导模型路由，让大模型选型更准更省
CARGO: A Framework for Confidence-Aware Routing of Large Language Models
- 基于嵌入的回归器预测模型表现，不确定时触发二分类器
- 在4个主流模型上实现76.4%的最优路由准确率，胜率72%-89%
- 无需人工标注，支持按任务类别定制路由，适合多模型部署
随着大语言模型规模、专长和延迟特征日益多样，如何将用户请求精准分配到最合适的模型，成为平衡性能与成本的关键挑战。本文提出CARGO（基于差距优化的类别感知路由框架），一种轻量级、置信度感知的动态模型选择方法。CARGO采用单一基于嵌入的回归器，通过模型自评的成对比较数据进行训练，以预测各模型表现，并在预测不确定性时调用一个可选的二分类器。该两阶段设计无需人类标注即可实现高精度、成本敏感的路由。为捕捉不同任务领域行为，CARGO还支持在数学、编程、推理、摘要和创意写作五个任务组上分别训练的类别专属回归器。在GPT-4o、Claude 3.5 Sonnet、DeepSeek V3和Perplexity Sonar共四个竞争性模型上评估，CARGO达到76.4%的最高单模型路由准确率，相较于个体专家模型的胜率范围为72%至89%。结果表明，基于置信度的轻量级路由可在极低开销下实现专家级性能，为实际多模型大语言模型部署提供可行方案。
原文摘要 · Abstract (English)
As large language models (LLMs) proliferate in scale, specialization, and latency profiles, the challenge of routing user prompts to the most appropriate model has become increasingly critical for balancing performance and cost. We introduce CARGO (Category-Aware Routing with Gap-based Optimization), a lightweight, confidence-aware framework for dynamic LLM selection. CARGO employs a single embedding-based regressor trained on LLM-judged pairwise comparisons to predict model performance, with an optional binary classifier invoked when predictions are uncertain. This two-stage design enables precise, cost-aware routing without the need for human-annotated supervision. To capture domain-specific behavior, CARGO also supports category-specific regressors trained across five task groups: mathematics, coding, reasoning, summarization, and creative writing. Evaluated on four competitive LLMs (GPT-4o, Claude 3.5 Sonnet, DeepSeek V3, and Perplexity Sonar), CARGO achieves a top-1 routing accuracy of 76.4% and win rates ranging from 72% to 89% against individual experts. These results demonstrate that confidence-guided, lightweight routing can achieve expert-level performance with minimal overhead, offering a practical solution for real-world, multi-model LLM deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。