用动态技能图提升智能体的多步任务组合能力
SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs
- 将技能表示为带关系的有向图，支持依赖与协同推理
- 在多个复杂任务上超越现有方法，尤其擅长多技能组合
- 适合需要长期规划与技能复用的自主智能体研究
技能库使大语言模型智能体能复用过往交互经验，但现有库大多将技能视为孤立条目，仅通过语义相似性检索，导致组合任务面临两大挑战：一是难以识别技能间的依赖与递进关系；二是缺乏结构线索，难以决定何时合并、拆分或删除技能。本文提出SKILLGRAPH框架，将可复用技能表示为有向图中的节点，类型化边编码先决条件、增强关系和共现关系。面对新任务时，该框架不仅检索单个技能，还生成有序技能子图，指导多步决策。图结构基于智能体轨迹和强化学习反馈持续更新，实现技能库与策略同步优化。在ALFWorld、WebShop及七个搜索增强型问答任务上的实验表明，SKILLGRAPH在记忆增强强化学习方法中达到领先性能，尤其在需多技能组合的复杂任务上表现显著提升。
原文摘要 · Abstract (English)
Skill libraries enable large language model agents to reuse experience from past interactions, but most existing libraries store skills as isolated entries and retrieve them only by semantic similarity. This leads to two key challenges for compositional tasks. Firstly, an agent must identify not only relevant skills but also how they depend on and build upon each other. Secondly, it also makes library maintenance difficult, since the system lacks structural cues for deciding when skills should be merged, split, or removed. We propose SKILLGRAPH, a framework that represents reusable skills as nodes in a directed graph, with typed edges encoding prerequisite, enhancement, and co-occurrence relations. Given a new task, SKILLGRAPH retrieves not just individual skills, but an ordered skill subgraph that can guide multi-step decision making. The graph is continuously updated from agent trajectories and reinforcement learning feedback, allowing both the skill library and the agent policy to improve together. Experiments on ALFWorld, WebShop, and seven search-augmented QA tasks show that SKILLGRAPH achieves state-of-the-art performance against memory-augmented RL methods, with especially large gains on complex tasks that require composing multiple skills.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。