让模型推理更快更准，训练时用理由引导，测试时直接出答案。
Dual-Head Reasoning Distillation: Improving Classifier Accuracy with Train-Time-Only Reasoning
- 训练时加两个头：一个分类头，一个学老师推理过程的推理头。
- 在七个任务上准确率提升0.65%到5.47%，尤其在因果推理任务上效果显著。
- 测试时关闭推理头，速度比传统推理快96到142倍，适合高效部署。
链式思维（CoT）提示常能提升分类准确率，但生成推理过程会带来显著吞吐下降（Wei等，2022；Cheng和Van Durme，2024）。为解决这一权衡，我们提出双头推理蒸馏（DHRD），一种适用于解码器仅语言模型（LMs）的简单训练方法：在训练中加入（i）一个用于训练与推理的池化分类头，以及（ii）一个由教师推理过程监督的推理头，仅在训练时使用。训练损失为标签交叉熵与输入+推理序列上的分词级语言模型损失的加权和。在七个SuperGLUE任务上，DHRD相比池化基线相对提升0.65%至5.47%，尤其在蕴含/因果类任务上增益更大。由于测试时禁用推理头，推理吞吐量与池化分类器相当，并在相同骨干模型下比CoT解码快96至142倍（QPS）。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) prompting often improves classification accuracy, but it introduces a significant throughput penalty with rationale generation (Wei et al., 2022; Cheng and Van Durme, 2024). To resolve this trade-off, we introduce Dual-Head Reasoning Distillation (DHRD), a simple training method for decoder-only language models (LMs) that adds (i) a pooled classification head used during training and inference and (ii) a reasoning head supervised by teacher rationales used only in training. We train with a loss function that is a weighted sum of label cross-entropy and token-level LM loss over input-plus-rationale sequences. On seven SuperGLUE tasks, DHRD yields relative gains of 0.65-5.47% over pooled baselines, with notably larger gains on entailment/causal tasks. Since we disable the reasoning head at test time, inference throughput matches pooled classifiers and exceeds CoT decoding on the same backbones by 96-142 times in QPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。