不靠提示词也能让大模型好好思考，靠的是自动打断提前结束。
"Well, Keep Thinking": Enhancing LLM Reasoning with Adaptive Injection Decoding
- 检测生成过程，发现快结束时自动插入提示词，逼模型继续推理
- 在多个推理任务上显著提升准确率，效果优于传统提示方法
- 适合想减少人工设计提示、提升模型自主推理能力的研究者
大型语言模型（LLMs）展现出强大的推理能力，通常归因于少样本或零样本思维链（CoT）提示。尽管有效，这些方法需要耗时的手动提示工程，引发一个问题：能否在不依赖显式提示的情况下诱导推理？本文提出一种新解码策略，无需显式提示即可激活LLM的推理能力。受零样本CoT和CoT解码启发，我们系统性地引导模型持续推理，防止过早终止。具体而言，监测模型生成过程，在其可能提前结束时注入指定短语以推动继续思考。在多种推理基准上的实验表明，该策略显著提升了LLM的推理表现，凸显基于解码干预作为传统提示技术替代方案的巨大潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit strong reasoning abilities, often attributed to few-shot or zero-shot chain-of-thought (CoT) prompting. While effective, these methods require labor-intensive prompt engineering, raising the question of whether reasoning can be induced without reliance on explicit prompts. In this work, we unlock the reasoning capabilities of LLMs without explicit prompting. Inspired by zero-shot CoT and CoT-decoding, we propose a novel decoding strategy that systematically nudges LLMs to continue reasoning, thereby preventing immature reasoning processes. Specifically, we monitor the model's generation and inject a designated phrase whenever it is likely to conclude its response prematurely, before completing the reasoning process. Our experimental evaluations on diverse reasoning benchmarks demonstrate that our proposed strategy substantially improves LLM reasoning capabilities, highlighting the potential of decoding-based interventions as an alternative to traditional prompting techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。