让大模型智能体自主创建、优化和选择技能，实现自我进化。
SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
- 通过轨迹反馈指导技能的生成与更新，让智能体从经验中学习。
- 在ALFWorld和WebShop上成功率分别提升8.8%和9.3%，超越现有方法。
- 适合研究自主智能体、强化学习与通用任务求解的开发者参考。
技能是提升大模型智能体处理复杂任务能力的有效机制，但当前框架中的技能创建、优化与选择通常依赖外部教师、人工规则或辅助模块。因此，技能仍是可调用的外部资源，而非智能体通过经验发展、适应并内化的自身能力。为赋予大模型智能体自主技能掌握能力，本文提出SkillMaster训练框架，使智能体能在任务执行中自主创造新技能、优化已有技能并选择累积技能。该能力基于三项关键设计：首先，通过轨迹感知的技能回顾训练，使智能体依据已完成任务的证据决定技能的提出、更新或保留；其次，每个候选技能修改均通过其在相关探测任务上的反事实效用来评估，提供直接学习信号；第三，引入DualAdv-GRPO，分别估计任务执行动作与技能编辑决策的优势，稳定联合训练。在ALFWorld和WebShop上的实验表明，SkillMaster相比最优基线分别提升总体成功率达8.8%和9.3%，表现最佳。进一步分析显示，经训练的智能体具备识别技能失败、从轨迹中提炼程序知识，并以极少技能库修改实现跨任务迁移的能力。总体而言，SkillMaster推动大模型智能体从单纯使用技能迈向能自主开发、调整与应用技能体系的自进化系统。
原文摘要 · Abstract (English)
Skills provide an effective mechanism for improving LLM agents on complex tasks, yet in existing agent frameworks, their creation, refinement, and selection are typically governed by external teachers, hand-designed rules, or auxiliary modules. As a result, skills remain external resources to be invoked, rather than capabilities that agents can develop, adapt, and internalize through experience. To endow LLM agents with autonomous skill mastery, we propose SkillMaster, a training framework that teaches agents to create new skills, refine existing skills, and select accumulated skills during task solving. This capability is achieved through three key designs. First, we train agents through trajectory-informed skill review, teaching agents to propose, update, or retain skills based on evidence from completed episodes. Second, each candidate skill edit is designed to be evaluated by its counterfactual utility on related probe tasks, providing a direct learning signal for training skill-editing decisions. Third, we introduce DualAdv-GRPO, which separately estimates advantages for task-solving actions and skill-editing decisions, stabilizing joint training across task solving and skill management. Experiments on ALFWorld and WebShop show that SkillMaster improves the overall success rate over state-of-the-art baselines by 8.8% and 9.3%, respectively, achieving the best performance among all compared methods. Further analysis reveals a marked shift in agent capability: agents trained with SkillMaster can identify skill failures, refine procedural knowledge from trajectory evidence, and transfer improvements to future tasks with limited skill-bank edits. Overall, SkillMaster moves LLM agents beyond mere skill use toward self-improving agents capable of developing, adapting, and applying their own skill repertoires.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。