arXiv:2510.09719cs.LGcs.AI2025-10AAAI被引 22

用上下文向量表示模型能力,实现无需重训的动态大模型路由。

ICL-Router: In-Context Learned Model Representations for LLM Routing

  • 通过上下文向量构建模型能力表示,匹配路由器语义空间。
  • 在分布内和分布外任务上均达到当前最优路由性能。
  • 新增模型可无缝接入,无需重训路由模块,适合多模型协同场景。

大型语言模型常表现出互补优势。模型路由通过动态将查询分配给最合适的模型来利用这些优势,但路由效果依赖于准确的模型表示,且添加新模型通常需重新训练,限制了可扩展性。为此,我们提出一种新型路由方法,使用上下文向量表示模型能力。该方法分两阶段:第一阶段,查询被嵌入并投影为向量,通过投影器与基于LLM的路由器联合训练以重建原始查询,使向量表示与路由器语义空间对齐;第二阶段,对候选模型在查询集上进行性能画像,路由器基于查询与模型表现的上下文向量预测模型能否正确回答新查询。大量实验表明,该方法在分布内与分布外任务中均达到当前最优路由性能。此外,该方法支持无须重训即可无缝集成新模型。代码已公开于 https://github.com/lalalamdbf/ICL-Router。

原文摘要 · Abstract (English)

Large language models (LLMs) often exhibit complementary strengths. Model routing harnesses these strengths by dynamically directing each query to the most suitable model, given a candidate model pool. However, routing performance relies on accurate model representations, and adding new models typically requires retraining, limiting scalability. To address these challenges, we propose a novel routing method using in-context vectors to represent model capabilities. The method proceeds in two stages. First, queries are embedded and projected into vectors, with a projector and LLM-based router trained to reconstruct the original queries, aligning vector representations with the router's semantic space. Second, each candidate model is profiled on a query set, and the router learns -- based on in-context vectors of query and model performance -- to predict whether each model can correctly answer new queries. Extensive experiments demonstrate that our method achieves state-of-the-art routing performance in both in-distribution and out-of-distribution tasks. Moreover, our method allows for seamless integration of new models without retraining the router. The code is available at https://github.com/lalalamdbf/ICL-Router.

模型路由上下文学习LLM协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。