让强化学习智能体持续验证并优化技能，提升长期决策能力
SkillForge: Evolving Verifiable Skills for Reinforcement Learning Agents
- 通过显式调用技能，让智能体在交互中同时优化动作与技能选择
- 在三个真实场景任务中，性能显著优于现有方法，证明持续验证的有效性
- 适合研究智能体长期记忆与可复用技能的学者和开发者
大型语言模型（LLM）智能体通过强化学习（RL）训练以完成复杂决策任务。然而，多数基于RL的智能体仍为片段式，无法在不同回合间积累可复用知识。近期基于技能的方法（如SkillRL）虽从原始轨迹中提取技能，但将技能库视为不可修改的追加式存储，未验证其有效性。本文提出SkillForge框架，实现技能的持续演化：通过环境交互对技能进行验证与优化。在智能体交互中显式体现技能使用，使强化学习能直接优化环境动作与技能调用决策。SkillForge引入基于证据的技能验证和多路径技能生成机制，确保技能库持续增长且质量不降。在ALFWorld、WebShop和AppWorld上的大量实验表明，SkillForge持续优于SkillRL，验证了持续验证技能在训练更强大LLM智能体中的有效性。
原文摘要 · Abstract (English)
Large language model (LLM) agents are trained with reinforcement learning (RL) for complex decision-making tasks. However, most RL-trained agents remain episodic and cannot accumulate reusable knowledge across episodes. Recent skill-based approaches, such as SkillRL, attempt to address this issue by extracting skills from raw trajectories, but treat the skill bank as an append-only repository without verifying whether stored skills remain effective. In this paper, we propose SkillForge, a framework for continuous skill evolution that enables skills to be verified and refined through environment interaction. By making skill usage explicit during agent interaction, RL can directly optimize both environment actions and skill invocation decisions. SkillForge further introduces evidence-based skill verification and multi-pathway skill induction, allowing the skill bank to continuously grow while maintaining its quality. Extensive experiments on ALFWorld, WebShop, and AppWorld show that SkillForge consistently outperforms SkillRL, demonstrating the effectiveness of continuously verified skills in training stronger LLM agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。