更强的模型无需复杂提示，简单零样本就能超越传统思维链提示效果。
Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve
- 用零样本直接提问，比给示例更有效
- 数学推理准确率从77%提升至84%
- 适合追求高效推理的开发者使用
思维链（CoT）提示仍是评估大语言模型推理能力的标准基线。最初该方法用于引导模型逐步推理，避免直接输出答案。然而，许多现代中等规模模型在面对推理任务时已能原生生成类似思维链的输出。我们在数学问题求解任务上测试多个现代模型，发现专门优化推理的模型在零样本设置下表现优于使用少样本思维链示例的设定，显著超越官方报告结果，且无需额外成本（如Mathstral在GSM8K上从~77%提升至~84%）。对于通用模型，零样本思维链提示也足以超越少样本基线。我们归因于“引导-干扰”权衡：标准思维链提示还需风格适配、格式合规及潜在的不必要上下文构建，反而分散模型对核心推理的注意力。随着模型能力增强，标准思维链提示正日益成为干扰源。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting remains the standard baseline for evaluating models' reasoning abilities. Originally, this technique was introduced to elicit step-by-step reasoning from large language models (LLMs), which would otherwise tend to directly output the final answer. However, many modern LLMs produce CoT-style responses \textit{natively} when presented with reasoning tasks, which made us revisit the effectiveness of standard CoT prompting. We evaluate several modern mid-sized language models on a math problem-solving task and find that models specialized for reasoning achieve better performance in a simple zero-shot setting than when using few-shot CoT examples - significantly surpassing officially reported results at no additional cost (e.g., from $\sim$77\% to $\sim$84\% for Mathstral on GSM8K). For the tested general-purpose model, a zero-shot CoT prompt is also sufficient to outperform a few-shot CoT baseline. We attribute this to a `guidance-distraction' tradeoff: standard CoT prompting also demands style adaptation, formatting compliance, and potentially undesired contextualization, which can distract models from the core reasoning task. Our findings suggest that using standard CoT prompting increasingly acts as a source of distraction as models grow stronger.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。