发现长思维链反而降低推理能力，提出最优思考时长策略
Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
- 用不同长度思维链教模型分阶段思考，自动匹配最优耗时
- 在数学任务上超越同类32B模型，接近教师模型表现
- 适合需要高效推理的复杂任务场景，如竞赛级数学
近期研究显示，延长大型语言模型（LLMs）的思维链（CoT）长度可显著提升复杂推理任务表现。然而，我们发现过度延长思维链可能在某些领域损害推理性能。在数学推理任务上的实验表明，不同领域存在不同的最优思维链长度分布。为此，我们提出一种思考最优缩放策略：首先用少量具有不同响应长度分布的种子数据，训练模型根据不同任务选择相应的推理投入；随后，模型在新问题上自选最短正确答案进行自我优化。基于Qwen2.5-32B-Instruct构建的自改进模型，在多个数学基准测试中优于其他基于蒸馏的32B o1类模型，并达到与生成种子数据的教师模型QwQ-32B-Preview相当的性能。
原文摘要 · Abstract (English)
Recent studies have shown that making a model spend more time thinking through longer Chain of Thoughts (CoTs) enables it to gain significant improvements in complex reasoning tasks. While current researches continue to explore the benefits of increasing test-time compute by extending the CoT lengths of Large Language Models (LLMs), we are concerned about a potential issue hidden behind the current pursuit of test-time scaling: Would excessively scaling the CoT length actually bring adverse effects to a model's reasoning performance? Our explorations on mathematical reasoning tasks reveal an unexpected finding that scaling with longer CoTs can indeed impair the reasoning performance of LLMs in certain domains. Moreover, we discover that there exists an optimal scaled length distribution that differs across different domains. Based on these insights, we propose a Thinking-Optimal Scaling strategy. Our method first uses a small set of seed data with varying response length distributions to teach the model to adopt different reasoning efforts for deep thinking. Then, the model selects its shortest correct response under different reasoning efforts on additional problems for self-improvement. Our self-improved models built upon Qwen2.5-32B-Instruct outperform other distillation-based 32B o1-like models across various math benchmarks, and achieve performance on par with the teacher model QwQ-32B-Preview that produces the seed data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。