arXiv:2606.03565cs.IR2026-06被引 2

让大模型智能匹配多个技能组合,提升任务执行成功率。

Skill Is Not Document: Query-Conditioned Compatibility for LLM Agent Skill Routing

论文配图:Skill Is Not Document: Query-Conditioned Compatibility for LLM Agent Skill Routing
图 1 · 摘自论文原文
  • 用查询相关性标注拒绝决策,构建更精准的技能匹配信号
  • 在新基准上达75.4%命中率,比最强基线提升36.6%
  • 适合研究智能体技能调度与多技能协同的开发者

大语言模型智能体依赖可复用的技能,技能检索成为系统关键组件。但技能检索不同于普通文档检索:理想的前K个结果不仅需单个相关,还需构成可执行的任务组合。现有基准和训练流程主要监督成对相关性,忽略语言模型判断技能组合不成立时产生的拒绝决策。本文提出R3-Skill,一个中英双语基准,保留这些拒绝作为查询条件下的兼容性监督信号。该基准包含10,246个去重技能、41,592个接受查询及32,828条拒绝标注,覆盖四个语言方向;所有多技能测试标签经多位专家独立审核,其中15,962条可解析拒绝被归入八类分类体系。我们进一步设计两阶段系统:R3-Embedding(多正样本双编码器用于大规模召回)与R3-Reranker(基于分级ListNet训练的交叉编码器)。分析表明该信号对交叉编码器重排序有帮助,但对双编码器目标无增益。在R3-Skill上,完整流水线达75.39% Hit@1、81.97% NDCG@10、33.27% Set-Compat,相对最强重排序基线提升36.6%。在SkillRet上也取得83.87% NDCG@10,证明具备跨数据集迁移能力。

原文摘要 · Abstract (English)

Large language model agents increasingly rely on reusable skills, making skill retrieval a critical front-end component of agent systems. Skill retrieval, however, is not ordinary document retrieval: a useful top-$K$ result must contain individually relevant skills that also form an executable set for the current query. Existing benchmarks and training pipelines largely supervise pairwise relevance and discard the rejection decisions produced when a language model judges a sampled skill combination to be implausible. We introduce R3-Skill, a Chinese--English benchmark that retains these rejections as query-conditioned compatibility supervision. R3-Skill contains 10,246 deduplicated skills, 41,592 accepted queries, and 32,828 rejected annotations across four language directions; all multi-skill test labels were independently reviewed by multiple experts, and 15,962 parseable rejections are organized into an eight-class taxonomy. We further propose a two-stage system composed of R3-Embedding, a multi-positive bi-encoder for large-pool recall, and R3-Reranker, a cross-encoder trained with graded ListNet supervision. Our analysis shows that this signal is stage-dependent, helping cross-encoder reranking while providing no benefit for the tested bi-encoder objective. On R3-Skill, the complete pipeline achieves $75.39\%$ Hit@1, $81.97\%$ NDCG@10, and $33.27\%$ Set-Compat, a $36.6\%$ relative gain over the strongest reranking baseline. It also obtains $83.87\%$ NDCG@10 on SkillRet, demonstrating transfer beyond R3-Skill.

智能体技能调度多技能检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。