让大模型自己生成推理链，用少量示范提升复杂任务表现
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst
- 用少量示例引导模型自动生成更长的推理链
- 在5个任务上平均提升超2.5分，64次采样时达+7.89分
- 适合追求推理能力增强的LLM研究者与应用开发者
推理阶段扩展受到广泛关注，通过延长思维链（Chain-of-Thought）显著提升大语言模型在复杂推理任务中的表现。这些更长的中间推理过程体现了人类认知中的元推理技能，如反思与分解，但难以人工构建。本文提出自推理语言模型（SRLM），使模型能自主生成更长的思维链，并通过自训练迭代优化性能。仅需1000个示范样本作为推理催化剂，即可引导模型从已有回答中挖掘隐藏推理路径。实验表明，SRLM不仅提升初始性能，且在后续迭代中实现更稳定、一致的改进。在两大基线模型上，五个推理任务（MMLU、GSM8K、ARC-C、HellaSwag、BBH）平均绝对提升超过+2.5分；当推理采样次数增至64次时，平均提升达+7.89分，揭示了SRLM中深度、多样且富有创造性的推理路径，优于强基线模型。
原文摘要 · Abstract (English)
Inference-time scaling has attracted much attention which significantly enhance the performance of Large Language Models (LLMs) in complex reasoning tasks by increasing the length of Chain-of-Thought. These longer intermediate reasoning rationales embody various meta-reasoning skills in human cognition, such as reflection and decomposition, being difficult to create and acquire. In this work, we introduce \textit{Self-Reasoning Language Model} (SRLM), where the model itself can synthesize longer CoT data and iteratively improve performance through self-training. By incorporating a few demonstration examples (i.e., 1,000 samples) on how to unfold hidden reasoning chains from existing responses, which act as a reasoning catalyst, we demonstrate that SRLM not only enhances the model's initial performance but also ensures more stable and consistent improvements in subsequent iterations. Our proposed SRLM achieves an average absolute improvement of more than $+2.5$ points across five reasoning tasks: MMLU, GSM8K, ARC-C, HellaSwag, and BBH on two backbone models. Moreover, it brings more improvements with more times of sampling during inference, such as absolute $+7.89$ average improvement with $64$ sampling times, revealing the in-depth, diverse and creative reasoning paths in SRLM against the strong baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。