arXiv:2601.09692cs.CLcs.AI2026-01ACL被引 2

无需标注数据,用生成内容训练智能模型路由系统。

Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection

  • 仅用生成的问答数据训练路由模型,摆脱对真实标注的依赖。
  • 生成数据质量差时,新方法比传统方法准确率高4.6%。
  • 适合缺乏标注数据或模型能力差异明显的实际部署场景。

大型语言模型(LLM)路由器可动态选择最优模型处理输入。现有方法通常依赖真实标注数据,但在用户请求分布异质且未知时难以获取。本文提出生成数据驱动的路由(RGD),即仅使用由生成器LLM根据任务描述生成的查询与答案进行训练。在四个不同基准和12个模型上评估了基于查询与标签的路由器和仅基于查询的路由器,发现随着生成器质量下降,前者性能衰减更快。分析揭示有效生成器需满足两个关键特征:能准确回答自身问题,且生成的问题能区分模型池中各模型的性能。通过筛选这些特征可提升生成数据质量。进一步提出CASCAL——一种新型仅基于查询的路由器,通过共识投票估计模型正确性,并利用分层聚类识别模型专属技能领域。在弱生成器数据上训练时,CASCAL相比最优查询-标签路由器绝对提升4.6%准确率。

原文摘要 · Abstract (English)

Large Language Model (LLM) routers dynamically select optimal models for given inputs. Existing approaches typically assume access to ground-truth labeled data, which is often unavailable in practice, especially when user request distributions are heterogeneous and unknown. We introduce Routing with Generated Data (RGD), a challenging setting in which routers are trained exclusively on generated queries and answers produced from high-level task descriptions by generator LLMs. We evaluate query-answer routers (using both queries and labels) and query-only routers across four diverse benchmarks and 12 models, finding that query-answer routers degrade faster than query-only routers as generator quality decreases. Our analysis reveals two crucial characteristics of effective generators: they must accurately respond to their own questions, and their questions must produce sufficient performance differentiation among the model pool. We then show how filtering for these characteristics can improve the quality of generated data. We further propose CASCAL, a novel query-only router that estimates model correctness through consensus voting and identifies model-specific skill niches via hierarchical clustering. CASCAL is substantially more robust to generator quality, outperforming the best query-answer router by 4.6% absolute accuracy when trained on weak generator data.

模型路由生成数据LLM无监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。