让大模型代理高效找对技能,靠的是理解技能内容而非仅看名字。
SkillRouter: Skill Routing for LLM Agents at Scale
- 用技能内容全文检索+重排序,提升路由准确率
- 在8万技能库中实现74.0%的顶级命中率
- 模型小5倍、速度快6倍,适合大规模部署
可复用的技能使大模型代理能将特定任务流程、工具能力与执行指引封装为模块化单元。随着技能库规模达数万级别,推理时暴露所有技能变得不可行,引发技能路由问题:需在规划或执行前精准识别相关技能。现有系统多采用渐进披露策略,仅展示技能名称和描述,隐藏完整实现体。我们在一个基于SkillsBench构建的基准上测试,包含约8万候选技能,发现隐藏技能体导致路由准确率下降37-44个百分点。更强的对照实验表明,缺失信号来自技能体内部,而非长度影响:通过体内容提炼的描述可部分弥补差距,但仍比全字段直接路由低7-21分;仅用元数据训练的编码器也比全字段模型低14.0分。为此我们提出SkillRouter,一个1.2B参数的轻量级体感知检索-重排序管道,在该基准上实现74.0%的Hit@1性能,优于所有对比基线,同时参数量减少13倍,运行速度提升5.8倍。排名优势在独立构建的额外基准上也得到验证。在四个编程代理的端到端实验中,路由提升带来任务成功率改善,且越强的代理获益越大。
原文摘要 · Abstract (English)
Reusable skills let LLM agents package task-specific procedures, tool affordances, and execution guidance into modular building blocks. As skill ecosystems grow to tens of thousands of entries, exposing every skill at inference time becomes infeasible. This creates a skill-routing problem: given a user task, the system must identify relevant skills before downstream planning or execution. Existing agent stacks often rely on progressive disclosure, exposing only skill names and descriptions while hiding the full implementation body. We examine this design choice on a SkillsBench-derived benchmark with approximately 80K candidate skills, targeting the practically important setting of large skill registries with heavy overlap. Across representative dense and reranking baselines on this setting, hiding the skill body causes a 37-44 percentage point drop in routing accuracy. Stronger controls show that the missing signal is body-resident rather than a simple length artifact: body-distilled descriptions recover part of the gap, but remain 7-21 points below direct all-field routing, while a metadata-only encoder trained with the same data remains 14.0 points below its all-field counterpart. Motivated by this finding, we present Skillrouter, a compact 1.2B body-aware retrieve-and-rerank pipeline. Skillrouter achieves 74.0% Hit@1 on our benchmark -- the strongest average top-1 routing performance among the baselines we evaluate -- while using 13$\times$ fewer parameters and running 5.8$\times$ faster than the strongest base pipeline. The ranking gains further generalize to a supplementary benchmark independently constructed from three skill sources. In a complementary end-to-end study across four coding agents, routing gains transfer to improved task success, with larger gains for more capable agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。