arXiv:2608.06196cs.AI2026-08

对比两种技能检索方法,发现结构化图谱无法提升效果。

Comparative Approaches to Agent Retrieval over Large Skill Libraries

  • 用混合检索器按需加载技能,效率高且覆盖广
  • 图谱在顶五命中率上比检索器低11.2分,效果更差
  • 图谱无法拓展检索范围,适合对结构有强依赖的场景

基于大型技能库的智能体需决定加载哪些技能及顺序。将整个库载入上下文成本高且缺乏自主排序结构。本文在690个技能的语料上比较两种方法:一种是结合词法与密集向量检索的混合排序器,实现稀疏按需加载;另一种是编码流程关系(如前置条件、数据流、顺序)的类型化知识图谱。在117个真实非重复查询上,混合排序器在顶五命中率达73.5% ± 8.0,约四分之一查询未被服务。当按设计使用(以图谱邻居替代额外排名结果,保持相同令牌预算),图谱表现显著更差(-11.2分,p=0.0007)。其由LLM生成的边层未能超越本地嵌入计算获得的邻居,73%排序器遗漏的查询根本无法通过图谱到达。原因在于预筛选拓扑限制:图谱候选边来自与排序器相同的嵌入邻域,98.6%的类型边连接的是排序器已覆盖的技能对。图谱可丰富关系语义,但无法扩展检索范围。进一步表明,作者自写查询评估会使顶五命中率虚高最高达44分,可能掩盖上述结果。贡献在于揭示为何增加结构不能提升强排序器性能,并识别出引入结构依赖最优化的条件。

原文摘要 · Abstract (English)

Agents backed by large skill libraries must decide which skills to load and in what order. Loading the entire library into context is expensive and provides no structure for autonomous sequencing. We study two systems for this problem over a corpus of 690 skills: a hybrid ranker combining lexical and dense-embedding retrieval for sparse, on-demand loading, and a typed knowledge graph encoding workflow relations such as prerequisites, data flow, and ordering. On a set of 117 realistic, non-echoing queries, the hybrid ranker retrieves the correct skill within the top five in 73.5% +/- 8.0 of cases, leaving roughly a quarter of queries unserved. When used as the design intended (substituting graph neighbours for additional ranked results at matched token budget), the graph is significantly worse (-11.2 points, p = 0.0007). Its LLM-generated edge layer adds nothing over neighbours obtained free from a local embedding pass, and 73% of the queries the ranker misses are not reachable through the graph at all. We attribute this to a pre-filter topology bound. Because the graph's candidate edges are drawn from the same embedding neighbourhood the ranker already searches, 98.6% of typed edges connect skills the ranker had already surfaced together. The graph can enrich relation semantics but cannot extend retrieval reach. We further show that evaluating on author-written queries overstates hit@5 by up to 44 points, which would have hidden these results entirely. Our contribution is a mechanistic account of why added structure does not improve retrieval over a strong ranker, and identify the conditions under which adding structural interdependence into the retrieval is optimal.

检索系统知识图谱智能体技能库

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。