用知识蒸馏提升小模型性能,适配边缘设备部署。
Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation
- 通过课程式微调+离线策略蒸馏增强小模型
- 百亿参数模型在边缘设备上达顶尖性能
- 适合资源受限场景的高效语言模型开发
大型语言模型虽显著提升了AI能力,但其庞大规模与高计算成本使其难以在资源受限的边缘环境直接部署。为此,本文提出一种系统化后训练流程,通过基于课程的监督微调(SFT)与离线策略知识蒸馏,有效提升小型模型精度。最终得到的指令微调模型在百亿参数级别中达到当前最优表现,在严格硬件约束下展现出强泛化能力,同时在多项任务中保持竞争力。该方法为在昇腾边缘设备上构建高性能语言模型提供了高效实用的解决方案。
原文摘要 · Abstract (English)
The rapid advancement of large language models (LLMs) has significantly advanced the capabilities of artificial intelligence across various domains. However, their massive scale and high computational costs render them unsuitable for direct deployment in resource-constrained edge environments. This creates a critical need for high-performance small models that can operate efficiently at the edge. Yet, after pre-training alone, these smaller models often fail to meet the performance requirements of complex tasks. To bridge this gap, we introduce a systematic post-training pipeline that efficiently enhances small model accuracy. Our post training pipeline consists of curriculum-based supervised fine-tuning (SFT) and offline on-policy knowledge distillation. The resulting instruction-tuned model achieves state-of-the-art performance among billion-parameter models, demonstrating strong generalization under strict hardware constraints while maintaining competitive accuracy across a variety of tasks. This work provides a practical and efficient solution for developing high-performance language models on Ascend edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。