让大模型学会用更短的推理步骤解题，节省计算资源。
Self-Training Elicits Concise Reasoning in Large Language Models
- 用自生成简洁推理路径进行微调，结合采样与少量示例引导。
- 在GSM8K和MATH上平均减少30%输出token，准确率不变。
- 适用于多种模型，尤其适合已过后训练的大模型使用。
链式思维（CoT）使大语言模型通过中间推理步骤解决复杂任务，但当前推理过程常含冗余内容，增加计算开销。我们分析发现，现有模型潜藏更简洁推理的能力。为此，提出基于自生成简洁路径的微调方法，利用最佳N次采样和少样本提示，在特定任务中引导模型。该方法在五个模型家族的GSM8K和MATH数据集上，平均减少30%输出token，同时保持平均准确率。通过利用大模型固有的随机性与上下文学习能力，该自训练策略在多种模型上均能稳健激发简洁推理。代码已开源：https://github.com/TergelMunkhbat/concise-reasoning。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning has enabled large language models (LLMs) to utilize additional computation through intermediate tokens to solve complex tasks. However, we posit that typical reasoning traces contain many redundant tokens, incurring extraneous inference costs. Upon examination of the output distribution of current LLMs, we find evidence on their latent ability to reason more concisely, relative to their default behavior. To elicit this capability, we propose simple fine-tuning methods which leverage self-generated concise reasoning paths obtained by best-of-N sampling and few-shot conditioning, in task-specific settings. Our combined method achieves a 30% reduction in output tokens on average, across five model families on GSM8K and MATH, while maintaining average accuracy. By exploiting the fundamental stochasticity and in-context learning capabilities of LLMs, our self-training approach robustly elicits concise reasoning on a wide range of models, including those with extensive post-training. Code is available at https://github.com/TergelMunkhbat/concise-reasoning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。