让大模型多轮思考，显著提升推理准确率
Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
- 用前一轮答案做提示，迭代优化推理过程
- QwQ-32B在AIME 2024上准确率从80.3%升至82.1%
- 方法简单通用，适合各类大模型增强推理能力
近期大型语言模型（如OpenAI-o1和DeepSeek-R1）证明了测试时扩展的有效性，即通过延长推理过程显著提升模型表现。然而，现有模型仍受限于长文本处理能力及强化学习训练效率。为此，本文提出一种简单而有效的测试时扩展方法——多轮思考（Multi-round Thinking）。该方法通过将前一轮的回答作为后续推理的提示，迭代优化模型推理。在多个模型（包括QwQ-32B和DeepSeek-R1）上的广泛实验表明，该方法在AIME 2024、MATH-500、GPQA-diamond和LiveCodeBench等多个基准上均取得性能提升。例如，QwQ-32B在AIME 2024上的准确率从第1轮的80.3%提升至第2轮的82.1%，DeepSeek-R1则从79.7%提升至82.0%。结果验证了多轮思考是一种普适且稳定的性能增强方式，具有广阔的未来应用前景。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated the effectiveness of test-time scaling, where extended reasoning processes substantially enhance model performance. Despite this, current models are constrained by limitations in handling long texts and reinforcement learning (RL) training efficiency. To address these issues, we propose a simple yet effective test-time scaling approach Multi-round Thinking. This method iteratively refines model reasoning by leveraging previous answers as prompts for subsequent rounds. Extensive experiments across multiple models, including QwQ-32B and DeepSeek-R1, consistently show performance improvements on various benchmarks such as AIME 2024, MATH-500, GPQA-diamond, and LiveCodeBench. For instance, the accuracy of QwQ-32B improved from 80.3% (Round 1) to 82.1% (Round 2) on the AIME 2024 dataset, while DeepSeek-R1 showed a similar increase from 79.7% to 82.0%. These results confirm that Multi-round Thinking is a broadly applicable, straightforward approach to achieving stable enhancements in model performance, underscoring its potential for future developments in test-time scaling techniques. The key prompt: {Original question prompt} The assistant's previous answer is: <answer> {last round answer} </answer>, and please re-answer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。