让大模型自己出题、自己解题，用极少数据实现能力跃升。
The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
- 模型自拟任务并尝试解决，通过自我评估选择挑战性任务。
- 在不足1.2%额外数据下，9个基准测试提升53.8%性能。
- 适合对数据效率与自主学习感兴趣的科研与工程人员。
强化学习（RL）在提升大语言模型（LLM）推理能力方面展现出潜力，但通常需要大量人工创建和标注数据。本文探索在极低数据依赖下通过强化学习改进LLM的方法。该方法交替让模型提出任务并尝试解决。为减少数据需求，提出两种基于自我意识的新机制：（1）自知难度预测，模型学习评估任务难度与自身能力的匹配度，优先选择有挑战但可解的任务；（2）自知极限突破，当任务超出自身能力边界时，主动请求外部数据以突破限制。在九个基准上的实验表明，仅需不到1.2%的额外数据，即实现53.8%的相对性能提升，验证了自知强化学习的有效性，并凸显自进化智能体训练的前景。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has demonstrated potential in enhancing the reasoning capabilities of large language models (LLMs), but such training typically demands substantial efforts in creating and annotating data. In this work, we explore improving LLMs through RL with minimal data. Our approach alternates between the LLM proposing a task and then attempting to solve it. To minimize data dependency, we introduce two novel mechanisms grounded in self-awareness: (1) self-aware difficulty prediction, where the model learns to assess task difficulty relative to its own abilities and prioritize challenging yet solvable tasks, and (2) self-aware limit breaking, where the model recognizes when a task is beyond its capability boundary and proactively requests external data to break through that limit. Extensive experiments on nine benchmarks showing a 53.8% relative improvement with less than 1.2% extra data demonstrate the efficacy of self-aware RL and underscore the promise of self-evolving agent training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。