让大模型在跨任务中自动提炼并复用技能,提升解题效率。
SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution

- 用统一策略交替解题与更新技能文档,实现跨任务技能进化。
- 在多个数据集上比最强基线提升2.3至8.5个百分点。
- 适合需要持续学习和跨任务迁移的智能体系统设计。
大型语言模型代理常面临相关但不同的任务,这些任务共享可复用的解决方案模式。然而,标准的代理强化学习将任务视为独立回合,现有技能学习方法要么仅针对单一任务重复尝试,要么采用多阶段流水线,导致提取、检索与执行过程纠缠。我们提出 SkillRise,一种统一的强化学习框架,用于跨任务学习技能。SkillRise 将相关实例组织为逐步增加难度的序列,并使用单一策略交替进行任务求解与维护动态演进的技能文档,直接传递给下一任务。通过解耦的信用分配机制,解题由当前任务结果监督,而技能整理则由折扣后的下游结果监督。在 ALFWorld、WebShop 与 ScienceWorld 上的实验表明,SkillRise 在 Pass@1 性能上优于对比方法,较最强基线提升 2.3 至 8.5 个百分点。尽管训练涉及不同任务,其学习到的技能整理策略仍对同一任务的重复尝试有效。进一步分析显示,测试时跨任务序列越长,性能越高,即使每项任务仅尝试一次。这表明 SkillRise 真正实现了跨任务技能迁移,而非依赖同一任务的重复采样。此外,SkillRise 在显著降低多阶段技能学习流水线运行开销的同时,仍保持优异性能。综合来看,该方法为大模型代理提供了一种简单高效的跨任务技能提取、优化与复用范式。
原文摘要 · Abstract (English)
Large language model agents often encounter related yet distinct tasks that share reusable solution patterns. Yet standard agentic reinforcement learning treats tasks as independent episodes, while existing approaches to skill learning either focus on repeated attempts of one task or use pipelines with multiple stages that entangle extraction, retrieval, and execution. We introduce SkillRise, a unified reinforcement learning framework for learning skills across tasks. SkillRise organizes related instances into progressively challenging sequences and uses a single policy to alternate between task solving and curating an evolving skill document passed directly to the next task. Decoupled credit assignment across tasks supervises solving with the current task outcome and curation with discounted downstream outcomes. Experiments on ALFWorld, WebShop, and ScienceWorld show that SkillRise achieves the strongest Pass@1 performance among the compared methods, with gains over the strongest baseline ranging from 2.3 to 8.5 percentage points. Although trained across distinct tasks, its learned curation policy remains effective for repeated attempts on the same task. Further analysis reveals scaling at test time across tasks: performance improves with longer sequences of related tasks even when each task is attempted only once. This trend suggests that SkillRise reuses transferable skills across tasks rather than benefiting from repeated sampling of the same task. SkillRise further retains strong performance while substantially reducing the runtime overhead of skill learning pipelines with multiple stages. Together, these results provide a simple and efficient training paradigm for LLM agents to extract, refine, and reuse transferable skills across tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。