用真实执行轨迹构建工具依赖图，提升大模型代理的工具排序准确率。
SkillGraph: Graph Foundation Priors for LLM Agent Tool Sequence Recommendation
- 基于49,831条成功轨迹构建有向加权依赖图，捕捉工具执行顺序规律。
- 在ToolBench上达Set-F1=0.271、Kendall-τ=0.096，API-Bank排序从负转正至+0.613。
- 分离候选检索与排序阶段，适合需高精度工具序列的应用场景。
大模型代理需从大型API库中选择并正确排序工具。现有方法仅依赖语义相似性进行检索和排序，但排序依赖工具间的实际数据依赖关系，而这些信息未体现在工具描述中，导致在结构化工作流领域可能出现负的Kendall-τ值。本文提出SkillGraph，一个从49,831条成功大模型代理轨迹中挖掘出的有向加权执行转移图，将工作流先后规律编码为可复用的图基础先验。在此基础上，设计两阶段解耦框架：GS-Hybrid用于候选选择，学习的成对重排序器用于排序。在ToolBench（9,965个测试实例；约16,000个工具）上，该方法达到Set-F1=0.271，Kendall-τ=0.096；在API-Bank上，Kendall-τ从-0.433提升至+0.613。在相同第一阶段输入下，该重排序器优于LLaMA-3.1-8B第二阶段重排序器。
原文摘要 · Abstract (English)
LLM agents must select tools from large API libraries and order them correctly. Existing methods use semantic similarity for both retrieval and ordering, but ordering depends on inter-tool data dependencies that are absent from tool descriptions. As a result, semantic-only methods can produce negative Kendall-$τ$ in structured workflow domains. We introduce SkillGraph, a directed weighted execution-transition graph mined from 49,831 successful LLM agent trajectories, which encodes workflow-precedence regularities as a reusable graph foundation prior. Building on this graph foundation prior, we propose a two-stage decoupled framework: GS-Hybrid retrieval for candidate selection and a learned pairwise reranker for ordering. On ToolBench (9,965 test instances; ~16,000 tools), the method reaches Set-F1 = 0.271 and Kendall-$τ$ = 0.096; on API-Bank, Kendall-$τ$ improves from -0.433 to +0.613. Under identical Stage-1 inputs, the learned reranker also outperforms LLaMA-3.1-8B Stage-2 rerankers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。