arXiv:2506.06923cs.AI2025-06被引 23

让大模型在生成时实时自我纠错,一次推理完成解题与验证。

Boosting LLM Reasoning via Spontaneous Self-Correction

  • 模型在单次推理中交替生成解法和验证,动态终止生成过程。
  • 在MATH500等数学基准上,小模型提升8.8%,大模型最高提升20%。
  • 适合需要高精度推理的场景,如数学竞赛题求解。

尽管大语言模型在众多任务上表现优异,数学推理仍是难点。现有自纠错方法将修正视为生成后的独立步骤,依赖额外提示和系统设计,无法实现真正的实时、自发纠错。为此,我们提出SPOC,一种在单次推理中实现解法与验证交替生成的自发自纠方法,根据验证结果动态终止生成,有效控制计算开销。SPOC从多智能体视角出发,让同一模型承担解法提出者与验证者双重角色。通过简单有效的合成数据微调,使模型具备自验证和多智能体协作能力,并借助在线强化学习进一步提升解法与验证准确率。在数学推理基准测试中,SPOC显著提升性能:在MATH500上,Llama-3.1-8B和70B Instruct模型分别提升8.8%和11.6%;在AMC23上分别提升10.0%和20.0%;在AIME24上分别提升3.3%和6.7%。

原文摘要 · Abstract (English)

While large language models (LLMs) have demonstrated remarkable success on a broad range of tasks, math reasoning remains a challenging one. One of the approaches for improving math reasoning is self-correction, which designs self-improving loops to let the model correct its own mistakes. However, existing self-correction approaches treat corrections as standalone post-generation refinements, relying on extra prompt and system designs to elicit self-corrections, instead of performing real-time, spontaneous self-corrections in a single pass. To address this, we propose SPOC, a spontaneous self-correction approach that enables LLMs to generate interleaved solutions and verifications in a single inference pass, with generation dynamically terminated based on verification outcomes, thereby effectively scaling inference time compute. SPOC considers a multi-agent perspective by assigning dual roles -- solution proposer and verifier -- to the same model. We adopt a simple yet effective approach to generate synthetic data for fine-tuning, enabling the model to develop capabilities for self-verification and multi-agent collaboration. We further improve its solution proposal and verification accuracy through online reinforcement learning. Experiments on mathematical reasoning benchmarks show that SPOC significantly improves performance. Notably, SPOC boosts the accuracy of Llama-3.1-8B and 70B Instruct models, achieving gains of 8.8% and 11.6% on MATH500, 10.0% and 20.0% on AMC23, and 3.3% and 6.7% on AIME24, respectively.

大模型数学推理自纠错推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。