通过解析技能文档结构生成更精准的伪查询,提升智能体技能检索效果。
Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrieval

- 构建技能知识图谱,分三步生成带参数一致性的伪查询
- 在4个基准上生成70万条多样化伪查询,召回率平均提升6.7个百分点
- 适用于技能检索、检索器训练及端到端智能体任务,适配多种大模型
伪查询生成可缓解智能体技能检索的监督瓶颈,但现有文档级方法通常忽略能力、参数与使用示例间的内在关联。为此,我们提出Skill2Query框架:先将技能文档解析为技能知识图谱,再通过风格模仿、查询模板生成和参数填充三阶段生成伪查询。这些查询可用于离线索引增强、在线查询扩展及检索器训练。在TheoremQA、LogicBench、ToolQA和CHAMP四个基准上,基于跨多个领域的近3万项技能,生成了70万条类别多样化的伪查询。Skill2Query在稀疏、密集及技能路由检索中均表现优异,平均Recall@1提升6.70个百分点。其生成的训练数据在各项指标上优于其他生成基线。多LLM后端验证表明,检索性能提升可直接转化为更高的智能体任务成功率。代码与资源见https://github.com/MatZaharia/Skill2Query。
原文摘要 · Abstract (English)
Pseudo-query generation can alleviate the supervision bottleneck for agent skill retrieval, but existing document-level approaches typically leave the rich internal relations among capabilities, parameters, and usage examples implicit. As a result, generated queries may be topically relevant to a skill while lacking capability grounding and parameter consistency, raising the question of whether explicitly exploiting a skill document's internal structure can produce more effective retrieval signals. We therefore propose Skill2Query, a framework that first parses a skill document into a Skill Knowledge Graph and then generates pseudo-queries through a three-stage process including style mimicking, query template generation, and parameter filling. The generated queries can be used for offline index augmentation, online query expansion, and retriever training. Four benchmarks (TheoremQA, LogicBench, ToolQA, and CHAMP) are used to evaluate Skill2Query with large-scale skill candidate pools across multiple downstream applications, including skill retrieval, retriever training, and end-to-end agent execution. Using nearly 30K skills across diverse domains, we generate 700K category-diverse pseudo-queries. Skill2Query consistently improves sparse, dense, and skill-routing retrieval, with an average Recall@1 gain of 6.70 percentage points across retrieval settings. Skill2Query-generated training data also achieves the best Recall@1 and nDCG@1 among the evaluated generation baselines. Further evaluations with multiple LLM backends demonstrate that improved skill retrieval translates into higher agent task success rates. Code and resources are available at https://github.com/MatZaharia/Skill2Query.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。