Eagle无需训练即可高效选择最适合的LLM，提升推理速度与准确性。
Eagle: Efficient Training-Free Router for Multi-LLM Inference
- 基于全局与局部ELO评分动态评估多模型能力，实现无训练路由。
- 在多个数据集上AUC最高提升23.52%，初始化耗时仅为基线1/20。
- 支持高并发在线更新，增量更新快100至200倍，适合实时服务场景。
大型语言模型（LLMs）能力各异、成本不同，催生了高效模型选择的需求。LLM路由器通过根据任务需求和预算约束动态选择最优模型来应对这一挑战。然而，现有方法在可扩展性和实时适应性方面存在不足，尤其在高负载在线环境中表现受限。本文提出Eagle，一种结合全局与局部ELO评分模块的新型LLM路由方法，可同时评估通用与专用模型能力，提供无训练、可扩展的解决方案，在提升模型选择质量的同时降低计算开销。实验表明，Eagle在多个数据集上持续优于基线方法，最高使AUC提升23.52%。此外，Eagle初始化仅需基线1/20时间，且在在线场景中增量更新速度快100至200倍，非常适合动态、高吞吐量的在线服务环境。
原文摘要 · Abstract (English)
The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need by dynamically choosing the most suitable model for a given query based on task requirements and budget constraints. However, existing routers face challenges in scalability and real-time adaptation, particularly in high-volume online environments. We present Eagle, a novel LLM routing approach that combines global and local ELO ranking modules to overcome these limitations. By evaluating both general and specialized LLM abilities, Eagle provides a scalable, training-free solution that enhances model selection quality while reducing computational overhead. Our experiments across multiple datasets show Eagle consistently outperforms baseline methods, with improvements of up to 23.52 percent in Area Under Curve (AUC) scores. Moreover, Eagle demonstrates remarkable efficiency, requiring only 1/20 of baseline methods' time for initialization and 100 to 200 times faster incremental updates in online scenarios, making it well-suited for dynamic, high-volume online serving environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。