用中间模型当老师,让小模型学会长串推理。
MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants
- 引入中间模型作教师助手,用中等长度思维链桥梁接能力与推理长度差距。
- 小模型在AIME2024等五个基准上平均分提升3.47至3.93分。
- 适合想部署高效推理小模型的研究者与开发者。
大语言模型(LLMs)在需要长思维链的规划、反思与优化任务中表现优异,但其庞大的模型规模和高计算开销难以广泛部署。相比之下,小语言模型(SLMs)因容量有限,难以学习长序列思维链,我们称之为“小模型可学性差距”。为解决此问题,我们提出MiCoTA框架——一种面向小模型的长思维链蒸馏方法。该框架利用中间尺寸模型作为教师助手,并采用中等长度的思维链序列,同时弥合容量与推理长度的双重差距。实验表明,尽管直接从大模型蒸馏的小模型表现不佳,但应用MiCoTA后显著提升推理性能:Qwen2.5-7B-Instruct 和 Qwen2.5-3B-Instruct 在 AIME2024、AMC、Olympiad、MATH-500 与 GSM8K 基准上平均得分分别提升 3.47 和 3.93。定量分析显示,该方法生成的数据更贴近基础小模型的分布,有助于理解其工作机制。研究为未来小模型长思维链数据蒸馏提供了新方向。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at reasoning tasks requiring long thought sequences for planning, reflection, and refinement. However, their substantial model size and high computational demands are impractical for widespread deployment. Yet, small language models (SLMs) often struggle to learn long-form CoT reasoning due to their limited capacity, a phenomenon we refer to as the "SLMs Learnability Gap". To address this, we introduce \textbf{Mi}d-\textbf{Co}T \textbf{T}eacher \textbf{A}ssistant Distillation (MiCoTAl), a framework for improving long CoT distillation for SLMs. MiCoTA employs intermediate-sized models as teacher assistants and utilizes intermediate-length CoT sequences to bridge both the capacity and reasoning length gaps. Our experiments on downstream tasks demonstrate that although SLMs distilled from large teachers can perform poorly, by applying MiCoTA, they achieve significant improvements in reasoning performance. Specifically, Qwen2.5-7B-Instruct and Qwen2.5-3B-Instruct achieve an improvement of 3.47 and 3.93 respectively on average score on AIME2024, AMC, Olympiad, MATH-500 and GSM8K benchmarks. To better understand the mechanism behind MiCoTA, we perform a quantitative experiment demonstrating that our method produces data more closely aligned with base SLM distributions. Our insights pave the way for future research into long-CoT data distillation for SLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。