让大模型在推理时按需纠错，减少无用计算。
Adaptive Rectification Sampling for Test-Time Compute Scaling
- 用提示词和评分模型控制纠错时机，只在必要步骤重思。
- 在GSM8K和MATH500上提升准确率，额外消耗token更少。
- 适合需要高效推理的复杂任务，如数学逻辑题。
新发布的OpenAI-o1和DeepSeek-R1表明，测试时扩展能显著提升模型性能，尤其在逻辑推理等复杂任务中。现有方法多通过生成更多或更长的思维链（CoTs）并自修正来实现，但自修正可能造成大量token浪费，且若推理已正确反而降低可读性。为证明大语言模型可在更细粒度层面纠正错误，我们提出自适应修正采样（AR-Sampling），能引导模型在合适步骤进行自修正。该方法利用过程监督奖励模型（PRM）作为验证器，并设计触发句来指导模型进行自适应的步级重思。在GSM8K和MATH500上的实验表明，该方法使模型以更精细的方式重思，提升解题准确率，同时生成合理的额外token数。
原文摘要 · Abstract (English)
The newly released OpenAI-o1 and DeepSeek-R1 have demonstrated that test-time scaling can significantly improve model performance, especially in complex tasks such as logical reasoning. Common test-time scaling methods involve generating more chains of thought (CoTs) or longer CoTs with self-correction. However, while self-correction can improve performance, it may lead to significant token waste and reduce readability of the CoT if the reasoning steps are already correct. To demonstrate that large language models (LLMs) can rectify errors at a more fine-grained level, we propose Adaptive Rectification Sampling (AR-Sampling), which can guide the LLMs to self-correction at the appropriate step. AR-Sampling leverages a process-supervised reward model (PRM) as a verifier and constructed trigger sentences to guide the model in adaptive step-level rethinking. Through the experiments on GSM8K and MATH500, it indicates that our approach enables the models to rethink in more fine-grained level, improving the accuracy of solutions, while generating a reasonable number of additional tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。