通过路由与分层推理,让大模型按需调用,提升效率。
Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
- 根据查询复杂度动态选轻量模型或逐级升级
- 减少计算开销,仅在必要时启用大模型
- 适合移动端、边缘设备等资源受限场景
近年来,语言模型(LM)在自然语言处理领域取得显著进展,在文本生成、摘要和问答等任务中表现优异。然而,其推理过程仍存在计算成本高、能耗大的问题,尤其在硬件、功耗或带宽受限的环境中部署困难。为应对这一挑战,近期方法提出多语言模型智能选择策略:根据查询复杂度动态分配计算资源——简单查询使用轻量模型,复杂查询则逐步升级至更大模型。本文综述了两种互补的高效大模型推理策略:(i) 路由机制,基于查询内容选择最合适的模型;(ii) 分层推理(HI),将查询依次传递给一系列模型,直到获得置信响应为止。两者均旨在通过优先使用轻量模型来降低计算开销。本文对比分析了两类技术在关键性能指标上的表现,梳理了基准测试进展,并指出了开放性挑战。最后,提出了未来研究方向,包括缩短响应时间、基于任务复杂度自适应选择模型,以及在异构环境中的可扩展部署,以推动大模型系统在真实场景中的高效与普及应用。
原文摘要 · Abstract (English)
Recent progress in Language Models (LMs) has dramatically advanced the field of natural language processing (NLP), excelling at tasks like text generation, summarization, and question answering. However, their inference remains computationally expensive and energy intensive, especially in settings with limited hardware, power, or bandwidth. This makes it difficult to deploy LMs in mobile, edge, or cost sensitive environments. To address these challenges, recent approaches have introduced multi LLM intelligent model selection strategies that dynamically allocate computational resources based on query complexity -- using lightweight models for simpler queries and escalating to larger models only when necessary. This survey explores two complementary strategies for efficient LLM inference: (i) routing, which selects the most suitable model based on the query, and (ii) cascading or hierarchical inference (HI), which escalates queries through a sequence of models until a confident response is found. Both approaches aim to reduce computation by using lightweight models for simpler tasks while offloading only when needed. We provide a comparative analysis of these techniques across key performance metrics, discuss benchmarking efforts, and outline open challenges. Finally, we outline future research directions to enable faster response times, adaptive model selection based on task complexity, and scalable deployment across heterogeneous environments, making LLM based systems more efficient and accessible for real world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。