用进化算法自动生成更优的数学推理步骤,提升模型表现。
CoTEvol: Self-Evolving Chain-of-Thoughts for Data Synthesis in Mathematical Reasoning

- 将推理过程看作种群演化,通过全局交叉和局部变异迭代优化。
- 合成正确推理路径成功率提升30%以上,训练后在8个数学基准上平均提高6.6%。
- 适合需要高质量推理数据的数学推理研究者,效率高且无需昂贵标注。
大型语言模型在具备高质量思维链(CoT)时展现出强大的数学推理能力,但高质量CoT的收集成本高昂。现有方法如从强模型蒸馏或基于测试时搜索的自生成,常面临收益递减或计算开销大的问题。本文提出CoTEvol,一种基于遗传进化的框架,将CoT生成视为对推理轨迹的群体搜索。候选轨迹通过轨迹层面的反射式全局交叉与步骤层面的不确定性引导局部变异进行迭代演化,实现整体重组与精细优化。设计轻量级、任务感知的适应度函数,引导演化向准确且多样化的推理方向发展。实验表明,CoTEvol在正确CoT生成成功率上提升超30%,显著增强结构多样性,并大幅提升效率。基于这些进化生成的CoT数据训练的LLM,在八个数学基准上平均性能提升6.6%,优于以往蒸馏与自生成方法。结果验证了进化式CoT生成在数学推理任务中的可扩展性与有效性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit strong mathematical reasoning when trained on high-quality Chain-of-Thought (CoT) that articulates intermediate steps, yet costly CoT curation hinders further progress. While existing remedies such as distillation from stronger LLMs and self-synthesis based on test-time search alleviate this issue, they often suffer from diminishing returns or high computing overhead.In this work, we propose CoTEvol, a genetic evolutionary framework that casts CoT generation as a population-based search over reasoning trajectories.Candidate trajectories are iteratively evolved through reflective global crossover at the trajectory level and local mutation guided by uncertainty at the step level, enabling holistic recombination and fine-grained refinement. Lightweight, task-aware fitness functions are designed to guide the evolutionary process toward accurate and diverse reasoning. Empirically, CoTEvol improves correct-CoT synthesis success by over 30% and enhances structural diversity, with markedly improved efficiency. LLMs trained on these evolutionary CoT data achieve an average gain of 6.6% across eight math benchmarks, outperforming previous distillation and self-synthesis approaches. These results underscore the promise of evolutionary CoT synthesis as a scalable and effective method for mathematical reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。