温度对大模型蒸馏效果有关键影响,高温度下正向KL反而优于反向KL。
Rethinking the Role of Temperature in Large Language Model Distillation

- 引入温度参数重新分析正向与反向KL的蒸馏差异
- 高温下正向KL显著提升,超越反向KL性能
- 该发现让传统简单方法也能媲美先进蒸馏技术
在大语言模型蒸馏中,反向KL(RKL)通常被认为优于正向KL(FKL),但这一结论常忽略温度τ的作用。本文重新审视温度的影响,发现其对两种方法具有不对称作用:温度能显著增强FKL中非主导词元的信号,而主要对RKL梯度进行缩放。因此,尽管在τ=1时RKL表现更好,但在更高温度下,FKL在指令跟随类基准上始终优于RKL。此外,温度不仅提升FKL,还改善更广泛的蒸馏目标,使简单的基于KL的方法达到与当前最先进方法相当的性能。
原文摘要 · Abstract (English)
Reverse Kullback-Leibler (RKL) divergence is widely favored over forward KL (FKL) in large language models (LLM) distillation, yet this preference is largely based on comparisons that omit the temperature $τ$, overlooking its central role in softening teacher distributions and improving knowledge transfer. In this work, we revisit temperature in LLM distillation and show that it fundamentally changes the comparison between FKL and RKL. Our analysis reveals an asymmetric effect: temperature substantially enriches FKL with non-dominant token signals, whereas it mainly rescales RKL gradients, causing FKL to benefit much more from $τ$ scaling than RKL. This asymmetry overturns the standard empirical conclusion: although RKL outperforms FKL at $τ=1$, FKL consistently surpasses RKL at higher temperatures across instruction-following benchmarks. Moreover, the impact of temperature is not limited to FKL; it improves a broader family of distillation objectives, enabling simple KL-based methods to achieve competitive performance against recent state-of-the-art LLM distillation approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。