arXiv:2608.12486cs.CL2026-08

让冻结的AI模型通过多样化技能演化自我提升,不改参数也能越用越聪明。

DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution

论文配图:DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution
图 1 · 摘自论文原文
  • 用多组技能并行演化,从任务经验中自动提炼可复用的推理和验证策略。
  • 在六类逻辑数学任务上超越现有方法,少量试错后性能提升显著且可跨模型迁移。
  • 适合想让大模型持续进化却无法更新参数的研究者与开发者。

大型语言模型(LLMs)无法在部署后保留经验而不更新参数。我们提出DIVE,一种基于多样性的框架,使冻结的LLM能通过任务经验与验证器反馈,演化持久的自然语言技能。这些技能包含可复用的推理流程、验证策略、常见错误模式与输出约束,由同一模型执行并修订,无需教师模型。由于自然语言技能演化是随机且非凸的搜索过程,单一轨迹优化易过拟合或陷入次优解。DIVE通过独立演化多组技能种群,经多样化变换自适应精炼,并联合选择互补技能集来缓解优化方差。在六项数学与逻辑推理任务及多个模型族中,DIVE持续优于现有推理方法、提示优化方案、技能发展框架与记忆基线。其仅需少量试运行即实现快速自我提升,性能增益远超参数微调(如SFT、GRPO)与提示优化(如GEPA),且技能可跨模型规模与家族迁移,使较小模型(如GPT-5-nano)在传统提示下达到甚至超越更大模型(如GPT-5)。结果表明,多样性驱动的技能演化是一种有效、可解释且无需参数更新的LLM自我提升路径。

原文摘要 · Abstract (English)

Large language models (LLMs) cannot retain post-deployment experience without parameter updates. We introduce DIVE, a diversity-driven framework that enables frozen LLMs to improve by evolving persistent natural-language skills from task experience and verifier feedback. These skills encode reusable reasoning procedures, verification strategies, common failure modes, and output constraints and are both executed and revised by the same underlying model without access to a teacher model. Since natural-language skill evolution is a stochastic, non-convex search process, optimizing a single skill trajectory can overfit to sampled experience or converge to a suboptimal solution. DIVE mitigates this optimization variance by independently evolving multiple skill populations from bootstrapped experience, adaptively refining them through diverse transformations, and jointly selecting a complementary set of skills. Across six mathematical and logical reasoning tasks and multiple model families, DIVE consistently outperforms existing reasoning methods, prompt-optimization approaches, skill-development frameworks, and memory-based baselines. It achieves rapid self-improvement from accumulated experience, obtaining substantially larger performance gains with fewer rollouts than parameter-based methods such as SFT and GRPO, and prompt optimization with GEPA. Further, the resulting skills transfer across model scales and families, enabling smaller models such as GPT-5-nano to match or outperform larger counterparts, i.e., GPT-5, under conventional prompting. These results establish diversity-driven skill evolution as an effective, interpretable, and parameter-free approach to LLM self-improvement.

自进化冻结模型技能演化零样本提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。