用高质量推理链提升小模型推理能力，减少所需推理链数量
Towards Efficient CoT Distillation: Self-Guided Rationale Selector for Better Performance with Fewer Rationales
- 基于学生模型自身表现筛选优质推理链，而非依赖数据量
- 在7个数据集上平均提升4.6%，用更少推理链实现更好效果
- 适合追求高效推理训练的AI研究者与工程师
链式思维（CoT）蒸馏旨在通过从大模型迁移多步推理能力来增强小语言模型（SLMs）的推理能力。然而，现有方法低估了推理链质量，主要关注数据数量，可能导致噪声或错误信息传给学生模型。为此，我们提出模型导向的推理链选择蒸馏（MoRSD），可识别并选择高质量推理链用于蒸馏以进一步提升性能。我们还提出推理难度（RD）度量，评估学生模型在给定推理链下生成正确答案的能力。相比基线，在三个任务的七个数据集上平均提升4.6%，且通过控制推理链的准确性、多样性和难度，使用更少推理链即达成目标。结果表明，少量高质量推理链即可显著提升学生模型的推理能力。该方法为高效CoT蒸馏提供了可行方案。代码将发布于https://github.com/Leon221220/MoRSD。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) distillation aims to enhance small language models' (SLMs) reasoning by transferring multi-step reasoning capability from the larger teacher models. However, existing work underestimates rationale quality, focusing primarily on data quantity, which may transfer noisy or incorrect information to the student model. To address the above issues, we proposed \textbf{M}odel-\textbf{O}riented \textbf{R}ationale \textbf{S}election \textbf{D}istillation (MoRSD), which can discern and select high quality rationales for distillation to improve performance further. We further propose a Rationale Difficulty (RD) metric to measure the ability of the student model to generate the correct answer under a given rationale. Compared to the baseline, we achieved 4.6$\%$ average improvement on seven datasets over three tasks, using fewer rationales by controlling their accuracy, diversity, and difficulty. Our results reveal that a small portion of the high quality rationales can enhance the reasoning ability of student models than the entire dataset. Our method promises to be a possible solution for efficient CoT distillation. Our code will be released in https://github.com/Leon221220/MoRSD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。