四智能体自进化框架，让大模型自动练推理，无需大量人工标注。
SAGE: Multi-Agent Self-Evolution for LLM Reasoning
- 四代理智体协同演化：挑战者出题，规划者列步骤，求解者答题，评议员评分过滤。
- 仅用少量初始数据，使Qwen-2.5-7B在代码和数学任务上提升超10%。
- 适合想提升大模型长程推理能力的研究者与工程师。
强化学习结合可验证奖励能提升大语言模型的推理能力，但多数方法仍依赖大规模人工标注数据。尽管自对弈可降低此依赖，却常缺乏明确规划与强质量控制，限制了长程多步推理的稳定性。我们提出SAGE（Self-evolving Agents for Generalized reasoning Evolution），一种闭环框架，由四个代理——挑战者、规划者、求解者与评议员——共享一个LLM主干，仅需少量种子数据即可协同演化。挑战者持续生成难度递增的任务；规划者将每项任务转化为结构化多步计划；求解者按计划作答，其正确性由外部验证器判定。评议员评估并过滤生成的问题与计划，防止课程漂移，保障训练信号质量，实现稳定自训练。在数学与代码生成基准测试中，SAGE在不同模型规模下均表现稳定，使Qwen-2.5-7B在LiveCodeBench上提升8.9%，在OlympiadBench上提升10.7%。
原文摘要 · Abstract (English)
Reinforcement learning with verifiable rewards improves reasoning in large language models (LLMs), but many methods still rely on large human-labeled datasets. While self-play reduces this dependency, it often lacks explicit planning and strong quality control, limiting stability in long-horizon multi-step reasoning. We present SAGE (Self-evolving Agents for Generalized reasoning Evolution), a closed-loop framework where four agents: Challenger, Planner, Solver, and Critic, co-evolve from a shared LLM backbone using only a small seed set. The Challenger continuously generates increasingly difficult tasks; the Planner converts each task into a structured multi-step plan; and the Solver follows the plan to produce an answer, whose correctness is determined by external verifiers. The Critic scores and filters both generated questions and plans to prevent curriculum drift and maintain training signal quality, enabling stable self-training. Across mathematics and code-generation benchmarks, SAGE delivers consistent gains across model scales, improving the Qwen-2.5-7B model by 8.9% on LiveCodeBench and 10.7% on OlympiadBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。