用对比学习提升大模型蒸馏效果，让小模型更像大模型。
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
- 针对师生模型设计不同损失函数，利用数据类型差异增强蒸馏
- 在指令遵循、代码生成等任务上显著提升学生模型性能
- 适用于对齐偏好、多模态扩展等场景，适用性强
尽管大语言模型蒸馏已取得成功，但以往方法对教师和学生生成的数据均采用相同损失函数，忽视了损失形式与数据类型的协同效应，导致学生模型性能提升不充分。为此，我们提出DistiLLM-2，一种对比学习方法，通过同时提高教师响应的似然度、降低学生响应的似然度，充分利用这种协同关系。大量实验表明，DistiLLM-2不仅在广泛任务（包括指令遵循、代码生成）上构建出高性能学生模型，还支持偏好对齐、视觉-语言扩展等多种应用。结果表明，对比策略能有效提升大模型蒸馏效率，实现师生模型在多样化数据上的精准对齐。
原文摘要 · Abstract (English)
Despite the success of distillation in large language models (LLMs), most prior work applies identical loss functions to both teacher- and student-generated data. These strategies overlook the synergy between loss formulations and data types, leading to a suboptimal performance boost in student models. To address this, we propose DistiLLM-2, a contrastive approach that simultaneously increases the likelihood of teacher responses and decreases that of student responses by harnessing this synergy. Our extensive experiments show that DistiLLM-2 not only builds high-performing student models across a wide range of tasks, including instruction-following and code generation, but also supports diverse applications, such as preference alignment and vision-language extensions. These findings highlight the potential of a contrastive approach to enhance the efficacy of LLM distillation by effectively aligning teacher and student models across varied data types.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。