通过分步检查点提升大模型数学推理能力，减少路径重复且更高效利用中间结果。
Stepwise Reasoning Checkpoint Analysis: A Test Time Scaling Method to Enhance LLMs' Reasoning
- 在推理每一步设置检查点，分组相似中间答案保持多样性。
- 利用所有中间结果进行最终决策，准确率在多个数据集上超越现有方法。
- 适合需要高可靠推理的场景，如自动解题与复杂逻辑任务。
基于思维链（Chain-of-Thought, CoT）的数学推理已成为大语言模型（LLMs）的重要能力，可通过测试时扩展（Test-Time Scaling, TTS）方法如束搜索和DVTS进一步提升。然而，这些方法在增加计算资源的同时，常导致路径同质化且未能有效利用中间结果。为此，我们提出分步推理检查点分析（Stepwise Reasoning Checkpoint Analysis, SRCA），在推理步骤间引入检查点。该框架包含两项核心策略：(1) 答案聚类搜索，按中间检查点答案对推理路径分组，在保证质量的同时维持多样性；(2) 检查点候选增强，将所有中间答案用于最终决策。SRCA有效缓解了路径同质化问题，并通过利用高质量中间结果构建容错机制。实验表明，相比现有TTS方法，SRCA在多个数学数据集上显著提升了推理准确率。
原文摘要 · Abstract (English)
Mathematical reasoning through Chain-of-Thought (CoT) has emerged as a powerful capability of Large Language Models (LLMs), which can be further enhanced through Test-Time Scaling (TTS) methods like Beam Search and DVTS. However, these methods, despite improving accuracy by allocating more computational resources during inference, often suffer from path homogenization and inefficient use of intermediate results. To address these limitations, we propose Stepwise Reasoning Checkpoint Analysis (SRCA), a framework that introduces checkpoints between reasoning steps. It incorporates two key strategies: (1) Answer-Clustered Search, which groups reasoning paths by their intermediate checkpoint answers to maintain diversity while ensuring quality, and (2) Checkpoint Candidate Augmentation, which leverages all intermediate answers for final decision-making. Our approach effectively reduces path homogenization and creates a fault-tolerant mechanism by utilizing high-quality intermediate results. Experimental results show that SRCA improves reasoning accuracy compared to existing TTS methods across various mathematical datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。