让小模型学会深度思考，通过逐步细化知识和自我反思提升推理能力
TinyThinker: Distilling Reasoning through Coarse-to-Fine Knowledge Internalization with Self-Reflection
- 分三阶段从粗到细引导小模型逐步内化知识
- 先学推理再自我反思，显著提升在常识推理任务上的表现
- 适合希望打造强推理能力的小模型研究者
大语言模型在多种任务中展现出出色的推理能力，促使人们尝试通过生成的推理数据将这些能力蒸馏到小型模型中。然而，直接训练于此类合成推理数据可能导致对推理过程的表面模仿，而非真正融合推理与底层知识。为此，我们提出 TinyThinker 框架，包含两项新方法：第一，采用三阶段流程，逐步引导学生模型经历推理过程，逐步细化知识粒度；第二，设计两阶段训练框架，先进行推理获取，再通过自生成数据进行自我反思。在常识推理基准测试中的实验表明，TinyThinker 性能优于基线。消融实验证实了各组件的有效性。我们预计该框架可扩展至其他知识密集型推理任务，为小型语言模型发展有效推理能力提供新策略。代码已开源。
原文摘要 · Abstract (English)
Large Language Models exhibit impressive reasoning capabilities across diverse tasks, motivating efforts to distill these capabilities into smaller models through generated reasoning data. However, direct training on such synthesized reasoning data may lead to superficial imitation of reasoning process, rather than fostering a genuine integration of reasoning capabilities with underlying knowledge. To address this, we propose TinyThinker, a framework introducing two novel approaches. First, we introduce a three-stage process that incrementally guides the student model through the reasoning process, progressively refining knowledge from coarse to fine granularity. Second, we develop a two-phase training framework comprising an initial reasoning acquisition phase followed by a self-reflection phase utilizing self-generated data. Experiments on commonsense reasoning benchmarks demonstrate that TinyThinker achieves superior performance compared to baselines. Ablation studies further validate the effectiveness of each component in our framework. We expect that TinyThinker can be extended to other knowledge-intensive reasoning tasks, offering an alternative strategy for developing effective reasoning capabilities in smaller language models. Codes are available at https://github.com/shengminp/TinyThinker
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。