arXiv:2604.02268cs.LG2026-04被引 63

让大模型学会内化技能,推理时无需调用外部工具

SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization

论文配图:SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
图 1 · 摘自论文原文
  • 通过渐进式去除非必要上下文,训练模型自主掌握技能
  • 在多个任务上提升9.7%~10.1%,每步上下文少于500词元
  • 适合需要零样本、低延迟推理的智能体应用

智能体技能作为可动态加载的程序性知识包,已成为增强大型语言模型智能体的有效方式。然而,推理时调用技能存在检索噪声、内容冗余和知识未真正内化的局限。本文提出SKILL0,一种用于技能内化的上下文强化学习框架。该方法在训练阶段采用动态课程学习:从完整技能上下文逐步撤回,将技能按类别分组并转化为包含交互历史的紧凑视觉上下文,教导模型工具调用与多轮任务完成。动态课程评估每个技能文件的策略收益,在线性衰减预算内保留仍有益的技能,最终实现完全零样本推理。大量实验表明,SKILL0在ALFWorld、Search-QA和WebShop任务上分别比标准强化学习基线提升9.7%、6.6%和10.1%,且每步上下文保持在0.5k token以内。代码已开源。

原文摘要 · Abstract (English)

Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching he model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file's on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7\% for ALFWorld, +6.6\% for Search-QA, and+10.1\% for WebShop), while maintaining a highly efficient context of fewer than 0.5k tokens per step. Our code is available at https://github.com/ZJU-REAL/SkillZero.

技能内化强化学习智能体零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。