让大模型按需切换,省钱又高效。
Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
- 根据问题难易动态选模型,不浪费算力。
- 组合多个模型可比单个最强模型还快还准。
- 适合部署多模型、追求效率的团队参考。
大型语言模型(LLMs)在能力、成本和领域上的多样化迅速增长,对推理时的智能模型选择提出了迫切需求。常规查询可用小模型解决,复杂任务则需要更强大的模型。然而静态部署无法根据输入查询的复杂性和领域灵活调整,导致性能不佳且成本上升。动态路由系统通过基于查询特征自适应选择模型,成为应对该挑战的有效方案。本文系统分析了多模型路由与级联方法,聚焦于推理时在一组独立训练的LLM间动态分配查询的系统。涵盖多种路由范式:查询难度、人类偏好、聚类、不确定性量化、强化学习、多模态及级联。对每种范式分析代表性方法并探讨关键权衡。除分类体系外,提出一个三维度概念框架:决策时机、使用信息、计算方式。该视角表明实际系统常为组合式,融合多种范式以适应运行约束。分析显示,有效的多模型路由需平衡多重目标,最优策略取决于部署与计算条件。设计良好的路由系统可通过战略性利用各模型专长,在提升效率的同时超越单一最强模型表现。但跨架构、模态和应用泛化的路由机制开发与评估仍面临开放挑战。
原文摘要 · Abstract (English)
The rapid growth of large language models (LLMs) with diverse capabilities, costs, and domains has created a critical need for intelligent model selection at inference time. While smaller models suffice for routine queries, complex tasks demand more capable models. However, static model deployment does not account for the complexity and domain of incoming queries, leading to suboptimal performance and increased costs. Dynamic routing systems that adaptively select models based on query characteristics have emerged as a solution to this challenge. This survey provides a systematic analysis of multi-LLM routing and cascading approaches, focusing on systems that route queries across a pool of independently trained LLMs at inference time. We cover diverse routing paradigms, including query difficulty, human preferences, clustering, uncertainty quantification, reinforcement learning, multimodality, and cascading. For each paradigm, we analyze representative methods and examine key trade-offs. Beyond taxonomy, we introduce a conceptual framework that characterizes routing systems along three dimensions: when decisions are made, what information is used, and how they are computed. This perspective highlights that practical systems are often compositional, integrating multiple paradigms under operational constraints. Our analysis demonstrates that effective multi-LLM routing requires balancing competing objectives. Choosing the optimal routing strategy depends on deployment and computational constraints. Well-designed routing systems can outperform even the most powerful individual models by strategically leveraging specialized capabilities across models while maximizing efficiency gains. Meanwhile, open challenges remain in developing and evaluating routing mechanisms that generalize across diverse architectures, modalities, and applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。