通过迭代采样提升测试时计算效率，让模型自纠错更有效。
Iterative Deepening Sampling as Efficient Test-Time Scaling
- 设计迭代深化采样框架，系统触发模型自纠错机制。
- 在Math500和AIME上显著提升难题解决成功率。
- 适合追求高效推理与模型自优化的研究者。
近期推理模型（如OpenAI的O1系列）在复杂推理任务上表现卓越，并揭示了新的测试时缩放规律。受此启发，众多研究致力于训练模型实现有效的自我评估与修正，以进一步推动缩放范式的发展。然而，如何从固定模型出发高效扩展测试时计算仍是一个挑战。本文聚焦于提升测试时复杂问题求解中自我反思数据生成的质量，从而间接改善下一代大语言模型的训练。我们探索了系统性触发模型自纠正机制对性能的提升效果，提出一种新颖的迭代深化采样算法框架，旨在增强自纠正能力并生成更高品质样本。在Math500和AIME基准上的大量实验表明，该方法在困难任务上取得更高成功率，并通过详尽的消融实验分析其在多种场景下的有效性。
原文摘要 · Abstract (English)
Recent reasoning models, such as OpenAI's O1 series, have demonstrated exceptional performance on complex reasoning tasks and revealed new test-time scaling laws. Inspired by this, many people have been studying how to train models to achieve effective self-evaluation and self-correction to further enable the scaling paradigm. However, less studied is how to efficiently scale test-time compute from a fixed model, and this remains a challenge. In this paper, we address this challenge by focusing on enhancing the quality of self-reflection data generation for complex problem-solving at test time, which can also subsequently improve the training of next-generation large language models (LLMs). Specifically, we explore how systematically triggering a model's self-correction mechanisms can improve performance on challenging reasoning tasks. To this end, we propose a novel iterative deepening sampling algorithm framework designed to enhance self-correction and generate higher-quality samples. Through extensive experiments on Math500 and AIME benchmarks, we demonstrate that our method achieves a higher success rate on difficult tasks and provide detailed ablation studies to analyze its effectiveness across diverse settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。