arXiv:2609.04865cs.AI2026-09

让推理与技能生成协同进化,提升智能体任务成功率。

CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution

论文配图:CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution
图 1 · 摘自论文原文
  • 构建推理与技能代理的协作框架,共享同一模型主干。
  • 在ALFWorld和WebShop上分别达98.4%和90.6%成功率,提升3.5和6.2个百分点。
  • 适合研究多智能体强化学习与技能自动演化的学者。

技能库通过让大语言模型智能体复用过程知识,提升代理强化学习的样本效率。然而现有范式存在结构缺陷:要么将技能演化与策略优化分离,要么将元技能固定为流程。二者均将技能视为被动管理对象,限制了技能的灵活演化及其与推理智能体的协同适应。为此,我们提出CoSkill,一个统一的多智能体强化学习框架,将静态的元技能流程重构为可学习的元技能智能体,并与推理智能体在层次化技能库上联合训练。通过将推理与元技能智能体建模为共享单一主干的协作团队,CoSkill实现端到端协同适应:推理智能体基于检索的任务技能和子集中的步骤技能决策,其任务表现则指导元技能智能体优化这些步骤技能。在ALFWorld和WebShop上的实验表明,CoSkill显著优于先前基于技能和强化学习的基线,在两个数据集上分别达到98.4%和90.6%的成功率(分别提升3.5和6.2个百分点)。如图1所示,CoSkill在早期样本效率、渐近性能和实际运行效率方面均表现更优。代码已公开于https://github.com/jinyuan-cookie/CoSkill。

原文摘要 · Abstract (English)

Skill libraries improve the sample efficiency of agentic reinforcement learning (RL) by enabling large language model (LLM) agents to reuse procedural knowledge. Yet existing paradigms exhibit structural shortcomings: they either decouple skill evolution from policy optimization or instantiate meta-skills as fixed workflows. Both treat skills as passive objects to be managed, limiting the flexible evolution of skills and their co-adaptation with the reasoning agent. To address the limitations, we propose CoSkill, a unified multi-agent RL framework that recasts the static meta-skill workflow as a learnable Meta-Skill Agent and jointly trains it with a Reasoning Agent over a hierarchical skill library. By modeling the Reasoning and Meta-Skill Agents as a cooperative team sharing a single backbone, CoSkill enables end-to-end co-adaptation: the Reasoning Agent conditions its actions on a retrieved task skill and step skills selected from its child set, while its task performance guides the Meta-Skill Agent in refining those step skills. Experiments on ALFWorld and WebShop show that CoSkill substantially outperforms prior skill-based and RL baselines, achieving success rates of 98.4% and 90.6%, respectively (+3.5 and +6.2 pp). As shown in Figure 1, CoSkill achieves superior early-stage sample efficiency, asymptotic performance, and wall-clock efficiency. Our code is available at https://github.com/jinyuan-cookie/CoSkill.

强化学习技能演化多智能体大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。