发现大模型路由存在性能瓶颈，提出突破方法。
The Routing Plateau: Understanding and Breaking the Accuracy Limits of LLM Routers
- 现有路由方法普遍依赖全局性能趋势，忽略具体查询特征。
- 21种方法在5个基准上均收敛于相近准确率，远低于理想路由。
- 提升数据量、编码器强度与端到端微调可突破性能极限。
大模型路由通过动态选择模型以优化服务的成本-质量权衡，已成为主流方案。我们对21种路由方法在五个基准上的广泛研究揭示了一个统一现象——路由平台：多数方法（包括kNN）表现相近，且显著低于理想路由（oracle router）。分析表明，该瓶颈主要源于可预测性限制：当前路由系统仅学习全局平均模型性能趋势，而非细粒度的查询特定信号。因此，它们能处理重叠的简单查询，却集体失败于需要实例级决策的困难查询。进一步研究显示，使用更大训练集、更强编码器及端到端微调可有效提升路由准确率。这些发现刻画了现有路由方法的共性局限，并为构建更高效的路由系统提供了可操作的指导。
原文摘要 · Abstract (English)
LLM routing has become a popular approach to improve the cost-quality trade-off of LLM services by dynamically selecting a model for each query. Recent work has explored a broad range of routing methods, including clustering-based routers, learned classifiers, pairwise ranking, and confidence-based approaches. Our extensive study of 21 routing methods across five benchmarks reveals a consistent phenomenon that we call the routing plateau: many methods, including kNN, achieve very similar accuracy and converge to a narrow performance range that remains far below the oracle router. Our investigation shows that the plateau is largely caused by a predictability bottleneck: current routers mainly learn global averaged model-performance trends rather than fine-grained query-specific routing signals. As a result, they solve overlapping easy queries but collectively fail on hard queries that require instance-specific routing decisions. We further study how to move beyond the plateau and find that larger training datasets, stronger encoders, and end-to-end fine-tuning can further improve routing accuracy. These findings characterize the common limits of current routing methods and provide insights and actionable directions for the community to build more effective routing systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。