arXiv:2605.00180cs.NIcs.CL2026-05被引 1

用公开技术报告构建大模型画像,实现冷启动路由与新模型快速集成。

RouteProfile: Graph-Based Profiling for Cold-Start LLM Routing

论文配图:RouteProfile: Graph-Based Profiling for Cold-Start LLM Routing
图 1 · 摘自论文原文
  • 基于技术报告中的元数据构建图结构模型画像。
  • 结构化画像在无训练冷启动路由中表现更优。
  • 新模型集成需画像与路由协同设计,适合模型管理场景。

大模型路由对根据用户需求和部署约束选择合适模型至关重要,但其效果依赖于对新查询和新模型的持续适应。新模型集成尤为困难,因缺乏用于路由器训练的查询-响应-奖励交互数据,也无法像新查询一样通过语义嵌入直接进行性能评估。现有画像方法受限:大模型生成的描述通常粗略,而基于交互的嵌入构建成本高昂。为此,我们提出 RouteProfile,一种基于图的画像框架,利用技术报告或模型卡片中的公开信号(如模型家族、描述、基准得分及领域)构建模型画像。该框架从组织形式、表示类型、聚合深度和学习配置四个维度研究画像构建。我们在免训练冷启动路由和新模型集成场景下进行评估,结果表明:(1) 结构化画像优于扁平基线;(2) 模型家族元数据比基准领域信息更可靠;(3) 有效的新模型集成需画像与路由器协同设计。整体表明,画像设计对路由系统适应动态模型生态至关重要。

原文摘要 · Abstract (English)

LLM routing is increasingly important for selecting suitable models under diverse user needs and deployment constraints, but its practical effectiveness depends on continual adaptation to emerging queries and newly released models. New-LLM integration is particularly challenging, as newly released models lack the query-response-reward interactions required for router training and cannot be profiled as directly as new queries via semantic embeddings. Existing profiles are limited: LLM-generated descriptions are often coarse, while interaction-based embeddings are costly to construct. To address this problem, we propose RouteProfile, a graph-based profiling framework that constructs LLM profiles from public signals in technical reports or model cards, including model family, model description, reported benchmark scores, and benchmark domains. RouteProfile organizes these heterogeneous signals into a graph and studies profile construction along four dimensions: organizational form, representation type, aggregation depth, and learning configuration. We evaluate RouteProfile in training-free cold-start routing and new-LLM integration settings. Experiments show that: (1) structured profiles outperform flat baselines in training-free cold-start routing; (2) model family metadata is more reliable than benchmark domain information; and (3) effective new-LLM integration requires profile-router co-design. Overall, our findings highlight the importance of profile design for enabling routing systems to adapt to the evolving model ecosystem.

大模型路由冷启动画像构建模型管理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。