用小模型模仿大模型，还能提升代码推理能力。
Knowledge Distillation for Large Language Models
- 大模型教小模型，再用思维链强化学习优化。
- 小模型在英语任务中保留70%-91%能力，西班牙语达95%。
- 适合部署在内存和算力有限的设备上。
我们提出一种资源高效的大型语言模型压缩框架，结合知识蒸馏与引导式思维链强化学习。以 Qwen 3B 为教师模型，Qwen 0.5B 为学生模型，在英文 Dolly-15k、西班牙文 Dolly-15k 以及代码 BugNet 和 PyTorrent 数据集上进行蒸馏，超参数在英文设置中调优以优化学生性能。在各类任务中，蒸馏后学生模型保持了教师模型的大量能力，同时体积显著减小：英文任务中保留率70%至91%，西班牙语高达95%，代码任务中 Rouge-L 达93.5%。对于代码任务，将思维链提示与 CoT-annotated Codeforces 数据上的 Group Relative Policy Optimization 结合，相比仅使用知识蒸馏，提升了推理连贯性和解题正确性。后续进行 4-bit 权重量化，进一步降低内存占用和推理延迟。结果表明，该方法可生成适用于资源受限场景的紧凑高效模型。
原文摘要 · Abstract (English)
We propose a resource-efficient framework for compressing large language models through knowledge distillation, combined with guided chain-of-thought reinforcement learning. Using Qwen 3B as the teacher and Qwen 0.5B as the student, we apply knowledge distillation across English Dolly-15k, Spanish Dolly-15k, and code BugNet and PyTorrent datasets, with hyperparameters tuned in the English setting to optimize student performance. Across tasks, the distilled student retains a substantial portion of the teacher's capability while remaining significantly smaller: 70% to 91% in English, up to 95% in Spanish, and up to 93.5% Rouge-L in code. For coding tasks, integrating chain-of-thought prompting with Group Relative Policy Optimization using CoT-annotated Codeforces data improves reasoning coherence and solution correctness compared to knowledge distillation alone. Post-training 4-bit weight quantization further reduces memory footprint and inference latency. These results show that knowledge distillation combined with chain-of-thought guided reinforcement learning can produce compact, efficient models suitable for deployment in resource-constrained settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。