用自然语言强化学习，让小模型自动学会可复用的任务技能。
SKILLER: Language-Level Reinforcement Learning for Reusable Skill Extraction in Small Language Models
- 用大模型当智能体和裁判，通过自然语言反馈训练小模型生成技能。
- 在多个评测中，小模型性能提升4.3%到20.4%，接近闭源大模型表现。
- 适合想低成本部署高效智能体的开发者，尤其关注小模型应用。
智能体技能是封装过程知识与领域专长的标准化形式，在智能体系统中作为行为约束机制，确保任务执行的可重复性与高质量。然而，当前主流的闭源模型如Codex和OpenClaw因推理成本过高，难以在真实场景中部署技能。开源模型在消费级显卡上快速迭代的能力，为降低此类开销提供了契机。但如何为小型模型自动生成高效适配的技能仍是难题。为此，我们提出SKILLER，一种以自然语言驱动的强化学习框架，针对小模型自动生成执行器特定技能。该框架利用强模型充当智能体与评判者，将小模型代理系统视为环境，并完全通过自然语言传播强化学习信号。在五个基准测试中，基于Qwen3.5-9B和Qwen3.5-4B的实验表明，SKILLER优于三种开源及一种闭源技能生成或演化方法，在9B模型上绝对性能提升4.3至20.4个百分点，在4B模型上提升1.8至13.3个百分点，且在SkillsBench单技能任务上性能媲美闭源强模型。项目地址：https://github.com/DANG-ai/SKILLER。
原文摘要 · Abstract (English)
Agent skills represent a standardized format for packaging procedural knowledge and domain expertise, serving within agent harness systems as an essential mechanism to continually constrain a language model's behavior space for repeatable, high-quality task execution. However, because strong closed-source models entail high inference costs, current popular agent harnesses, such as Codex and OpenClaw, remain prohibitively expensive when deploying these skills to accomplish real-world tasks. The rapid capability enhancement of open-source models deployable on consumer-grade GPUs presents a compelling opportunity to drastically reduce these costs by leveraging skill-based behavioral constraints. Nevertheless, automatically generating effective skills tailored specifically for such compact models remains a significant practical challenge. To address this, we propose SKILLER, a natural-language-driven reinforcement learning framework designed to automatically generate executor-specific skills for small models, which employs a strong model as the actor and critic, treats the small-model agent system as the environment, and propagates all reinforcement learning signals entirely via natural language. Extensive experimental evaluations across five relevant benchmarks using Qwen3.5-9B and Qwen3.5-4B demonstrate that SKILLER outperforms three open-source and one closed-source skill generation or evolution methods, achieving absolute gains ranging from 4.3 to 20.4 percentage points for the 9B model and 1.8 to 13.3 points for the 4B model, while remarkably matching the performance of strong closed-source models on single-skill tasks in SkillsBench. The project is available at https://github.com/DANG-ai/SKILLER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。