让智能体通过自动学习技能并不断进化，提升复杂任务表现。
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
- 从经验中自动提取可复用的高层次技能，构建技能库。
- 在多个任务上性能超越基线15.3%以上，且随任务复杂度提升仍稳定。
- 适合需要长期积累与迭代优化的AI代理系统研究者。
大型语言模型（LLM）智能体在复杂任务中表现卓越，但常孤立运行，无法有效利用过往经验。现有基于记忆的方法多存储原始轨迹，冗余且噪声大，难以提取可复用的行为模式。本文提出SkillRL框架，通过自动技能发现与递归演化，连接原始经验与策略改进。该方法引入基于经验的提炼机制，构建层次化技能库SkillBank，采用自适应检索策略获取通用与特定任务启发式，并设计递归演化机制，使技能库与智能体策略协同进化。实验在ALFWorld、WebShop及七个搜索增强任务上验证，结果表明SkillRL达到当前最优性能，相比强基线平均提升超15.3%，且在任务复杂度增加时保持鲁棒性。代码已开源。
原文摘要 · Abstract (English)
Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。