用线性模型实现快速低成本的专家大模型路由，支持动态增删。
IR3DE: A Linear Router for Large Language Models
- 基于岭回归构建线性路由，无需复杂训练。
- 推理任务上性能达98.4%标准化得分，超越基线。
- 新增或移除专家模型无需重训路由器，适合动态部署。
基础大语言模型（LLM）在各类通用任务中表现优异，并可通过领域专家型LLM在特定任务上取得显著成果。随着可用LLM数量持续增长，推理路由机制被提出以针对每个提示选择最合适的模型。然而，现有路由方法要么仅优化从弱到强通用模型的成本，要么需要大量训练来支持领域专家路由。本文提出IR3DE，一种基于岭回归的领域专家路由机制，可为每个提示提供廉价且快速的路由决策。我们在两种因果语言建模（CLM）设置下评估IR3DE：一种是所有领域的下一个词预测任务，另一种是各领域具有独立推理任务的设置。尽管是线性路由，IR3DE在两种CLM设置中的表现与基线相当，在推理设置中更胜一筹，达到98.4%的标准化性能。此外，IR3DE支持在不重新训练路由器的前提下添加或移除新领域专家模型，实现对可变模型集合的低扰动服务。代码已开源：github.com/gensyn-ai/IR3DE。
原文摘要 · Abstract (English)
Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs. With the ever-growing list of available LLMs, inference routers are being proposed to select the most appropriate LLM for each prompt. However, existing routing methods either optimize cost across weak-to-strong generalist LLMs or require substantial training to support domain-expertise routing. In this paper, we propose IR3DE, a Ridge Regression-based Router for Domain Experts that provides cheap and fast routing decisions for each prompt. We evaluate IR3DE in two Causal Language Modeling (CLM) settings where the tasks are next-token prediction for all domains, and one reasoning setting where each domain has its own distinct reasoning task. Despite being a linear router, IR3DE achieves performance comparable to the other baselines in both CLM settings, and surpassing them in the reasoning setting, with a normalized performance of 98.4%. Moreover, IR3DE enables the addition or removal of new domain experts without requiring the router to be retrained from scratch, allowing a dynamic set of LLMs to be served with minimal disruption to the router itself. Our code is available at: github.com/gensyn-ai/IR3DE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。