CoT提示能显著提升大模型推理能力,减少过度思考。
Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinking
- 用CoT提示引导大模型,控制思维过程长度
- 小模型在简单任务上增益明显,大模型在复杂任务上提升更大
- 一-shot CoT比多样本更优,有效缓解模型对反思词的过拟合
近期大型语言模型(LLMs)发展出具备反思与自修正能力的推理型大模型(RLLMs),展现出测试时扩展的有效性。本文首次系统分析零样本和少样本CoT提示对不同规模RLLMs在数学推理任务上的影响。实验覆盖1.5B至32B参数模型,发现尽管存在担忧,但CoT提示在多数场景下显著提升性能。大模型在简单任务上增益小,但在复杂问题上提升显著;小模型则相反。进一步分析显示,CoT提示可有效控制思考标记数与推理步数分布,使过度反思减少约90%。注意力日志分析表明,RLLMs易过拟合于反思相关词汇,而外部CoT引导可缓解此问题。值得注意的是,一-shot CoT始终优于少样本方法。研究为优化RLLMs性能提供了关键提示策略指导。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have introduced Reasoning Large Language Models (RLLMs), which employ extended thinking processes with reflection and self-correction capabilities, demonstrating the effectiveness of test-time scaling. RLLMs exhibit innate Chain-of-Thought (CoT) reasoning capability obtained from training, leading to a natural question: "Is CoT prompting, a popular In-Context Learning (ICL) method for chat LLMs, necessary to enhance the reasoning capability of RLLMs?" In this work, we present the first comprehensive analysis of the impacts of Zero-shot CoT and Few-shot CoT on RLLMs across mathematical reasoning tasks. We examine models ranging from 1.5B to 32B parameters, finding that contrary to concerns, CoT prompting significantly enhances RLLMs' performance in most scenarios. Our results reveal distinct patterns: large-capacity models show minimal improvement on simple tasks but substantial gains on complex problems, while smaller models exhibit the opposite behavior. Further analysis demonstrates that CoT prompting effectively controls the distribution of the numbers of thinking tokens and reasoning steps, reducing excessive reflections by approximately 90% in some cases. Moreover, attention logits analysis reveals the RLLMs' overfitting to reflection-related words, which is mitigated by external CoT guidance. Notably, our experiments indicate that for RLLMs, one-shot CoT consistently yields superior performance compared to Few-shot CoT approaches. Our findings provide important insights for optimizing RLLMs' performance through appropriate prompting strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。