让小模型自己发现并激活隐藏的推理能力，提升逻辑推理效果。
Self-Enhanced Reasoning Training: Activating Latent Reasoning in Small Models for Enhanced Reasoning Distillation
- 通过自训练筛选并强化小模型自生成的推理路径，激活潜在推理能力。
- 在零样本条件下，使GPT-2类小模型在推理任务上性能显著提升。
- 适合希望低成本部署强推理能力的小模型研究者使用。
大语言模型（LLMs）的快速发展显著提升了其推理能力，可完成复杂任务。然而，计算效率更高的小型模型（如GPT-2）往往缺乏此类能力。尽管已有研究通过推理蒸馏帮助小模型学习推理，但多数方法仅聚焦于优化教师模型生成的推理路径。我们观察到，小模型在采样过程中能生成高质量的推理路径，即使未使用思维链提示，这些路径因概率过低而在标准解码策略下难以显现。为此，我们提出自增强推理训练（SERT），通过在零样本条件下对自我生成的推理路径进行过滤与自训练，激活小模型中的潜在推理能力。实验以OpenAI的GPT-3.5为教师模型，GPT-2为学生模型，结果表明SERT显著提升了小模型的推理表现，增强了推理蒸馏的效果。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) has significantly enhanced their reasoning abilities, enabling increasingly complex tasks. However, these capabilities often diminish in smaller, more computationally efficient models like GPT-2. Recent research shows that reasoning distillation can help small models acquire reasoning capabilities, but most existing methods focus primarily on improving teacher-generated reasoning paths. Our observations reveal that small models can generate high-quality reasoning paths during sampling, even without chain-of-thought prompting, though these paths are often latent due to their low probability under standard decoding strategies. To address this, we propose Self-Enhanced Reasoning Training (SERT), which activates and leverages latent reasoning capabilities in small models through self-training on filtered, self-generated reasoning paths under zero-shot conditions. Experiments using OpenAI's GPT-3.5 as the teacher model and GPT-2 models as the student models demonstrate that SERT enhances the reasoning abilities of small models, improving their performance in reasoning distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。