arXiv:2510.17498cs.CL2025-10被引 2

小模型通过自我迭代推理,突破了难题解答极限。

Deep Self-Evolving Reasoning

  • 将推理过程建模为马尔可夫链,利用微弱改进概率持续演化。
  • 在AIME 2024-2025上解决9道题中的5道未解难题,超越600B参数教师模型。
  • 适合研究自进化推理机制或提升小模型推理能力的学者。

长链条思维推理已成为大语言模型高级推理的核心。尽管近期验证-修正框架使专有模型能解决奥数级问题,但其效果依赖于强而可靠的验证与修正能力,这在开源、小规模模型中仍较脆弱。本文提出一种称为深度自演化推理(DSER)的概率范式,即使在困难任务上验证与修正能力较弱,也能显著扩展此类模型的推理能力。我们将迭代推理视为马尔可夫链,每一步代表解空间中的随机转移。关键洞察是:只要改进概率略高于退化概率,正确解收敛即被保证。通过并行运行多个长时程自演化过程,DSER放大微小正向趋势,使模型渐近逼近正确答案。实证上,将DSER应用于DeepSeek-R1-0528-Qwen3-8B模型,在挑战性AIME 2024-2025基准上解决了9道题中的5道此前无法解决的问题,并通过多数投票提升整体性能,使其单轮准确率超过600B参数教师模型。除测试时扩展实用价值外,该框架还用于诊断当前开源推理器的根本局限——在自验证、自修正与稳定性方面的不足,为下一代具备内在自演化能力模型的发展指明方向。

原文摘要 · Abstract (English)

Long-form chain-of-thought reasoning has become a cornerstone of advanced reasoning in large language models. While recent verification-refinement frameworks have enabled proprietary models to solve Olympiad-level problems, their effectiveness hinges on strong, reliable verification and correction capabilities, which remain fragile in open-weight, smaller-scale models. This work demonstrates that even with weak verification and refinement capabilities on hard tasks, the reasoning limits of such models can be substantially extended through a probabilistic paradigm we call Deep Self-Evolving Reasoning (DSER). We conceptualize iterative reasoning as a Markov chain, where each step represents a stochastic transition in the solution space. The key insight is that convergence to a correct solution is guaranteed as long as the probability of improvement marginally exceeds that of degradation. By running multiple long-horizon, self-evolving processes in parallel, DSER amplifies these small positive tendencies, enabling the model to asymptotically approach correct answers. Empirically, we apply DSER to the DeepSeek-R1-0528-Qwen3-8B model. On the challenging AIME 2024-2025 benchmark, DSER solves 5 out of 9 previously unsolvable problems and boosts overall performance, enabling this compact model to surpass the single-turn accuracy of its 600B-parameter teacher through majority voting. Beyond its immediate utility for test-time scaling, the DSER framework serves to diagnose the fundamental limitations of current open-weight reasoners. By clearly delineating their shortcomings in self-verification, refinement, and stability, our findings establish a clear research agenda for developing next-generation models with powerful, intrinsic self-evolving capabilities.

推理增强自演化小模型数学推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。