用中间模型提升小模型的多步推理能力
Mentor-KD: Making Small Language Models Better Multi-step Reasoners
- 引入中间导师模型生成高质量推理路径和软标签
- 在多个复杂任务上使小模型推理准确率显著提升
- 适合希望提升小模型推理能力的研究者与工程师
大型语言模型（LLMs）通过链式思维（CoT）提示在多种复杂任务中表现出色。近期研究提出知识蒸馏方法——推理蒸馏，通过微调生成由大模型教师产生的多步推理路径，将推理能力迁移至小型语言模型。然而，现有方法在数据质量与软标签提供方面存在不足。本文提出Mentor-KD，通过引入一个中间尺寸的任务特定微调模型作为导师，在推理蒸馏过程中增强推理标注并提供软标签，有效提升小型模型的多步推理能力。大量实验验证了该方法在多种模型和复杂推理任务中的有效性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have displayed remarkable performances across various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently, studies have proposed a Knowledge Distillation (KD) approach, reasoning distillation, which transfers such reasoning ability of LLMs through fine-tuning language models of multi-step rationales generated by LLM teachers. However, they have inadequately considered two challenges regarding insufficient distillation sets from the LLM teacher model, in terms of 1) data quality and 2) soft label provision. In this paper, we propose Mentor-KD, which effectively distills the multi-step reasoning capability of LLMs to smaller LMs while addressing the aforementioned challenges. Specifically, we exploit a mentor, intermediate-sized task-specific fine-tuned model, to augment additional CoT annotations and provide soft labels for the student model during reasoning distillation. We conduct extensive experiments and confirm Mentor-KD's effectiveness across various models and complex reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。