arXiv:2607.22529cs.CL2026-07被引 1

让大模型通过技能自演进,兼顾任务多样与验证可靠。

Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills

论文配图:Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
图 1 · 摘自论文原文
  • 用动态技能控制实现任务生成与求解协同进化。
  • 在工具使用和推理基准上显著提升模型性能。
  • 适合想提升模型泛化能力的研究者与开发者。

大模型训练正从人工设计转向交互驱动的自我演化。现有方法面临任务多样性与验证可靠性之间的根本矛盾:环境受限的方法反馈精确但领域狭窄,开放生成则任务广却缺乏可靠验证,易导致错误奖励污染训练循环。本文将智能体技能视为解决该矛盾的关键:每个技能确保特定场景下的深度可验证执行,而动态技能路由维持任务多样性。基于此,提出技能自演进(Skill-SP)框架,包含提议者、求解者与动态技能控制器。三者通过强化学习闭环协同演化:提议者根据动态采样的技能生成挑战性任务;求解者探索候选解以突破能力边界;技能控制器收集执行反馈,更新并扩展技能库。实证表明,Skill-SP作为稳健演化引擎,能持续推动成熟基线模型的性能上限,并显著逆转初始不匹配模型的表现。代码已开源。

原文摘要 · Abstract (English)

LLM training is shifting from manual design and annotation to interaction-driven self-evolution. However, existing self-evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment-bound methods obtain precise feedback but confine learning to narrow domains, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop. We identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open-ended task variety. Leveraging this insight, we introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a dynamic skill controller. Orchestrated via a reinforcement learning loop, these components co-evolve in a continuous self-play loop: the proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library. This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration. Empirical evaluations on tool-use and reasoning benchmarks demonstrate that Skill-SP, serving as a robust evolution engine, consistently pushes the performance ceiling of competent backbones while catalyzing striking turnarounds for initially misaligned models. Our code is available at https://github.com/Qwen-Applications/skill-self-play.

大模型演化技能机制自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。