小模型也能高效推荐，压缩技术让大模型能力落地
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems
- 用知识蒸馏和结构化剪枝量化，压缩大模型为小模型
- 在真实社交平台中保持高精度，推理速度提升数倍
- 适合需要低延迟、低成本部署的推荐系统场景
大型语言模型（LLMs）在搜索、推荐系统等工业应用中表现卓越。尽管模型越大性能越优，但其巨大计算需求使其难以大规模实际应用。本文提出一套完整的小语言模型（SLMs）训练与部署方案，聚焦知识蒸馏与结构化剪枝结合量化压缩技术，使小型模型在保留大模型核心能力的同时，显著降低训练与服务成本及延迟。我们在一个大型专业社交网络平台的多种应用场景中验证了该方法，展示了其在预测与推理类推荐任务中的有效性，并分享了硬件优化策略，进一步提升了服务速度与吞吐量。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Although scaling laws indicate that larger models generally yield better generalization and performance, their substantial computational requirements often render them impractical for many real-world scenarios at scale. In this paper, we present a comprehensive set of insights for training and deploying small language models (SLMs) that deliver high performance for a variety of industry use cases. We focus on two key techniques: (1) knowledge distillation and (2) model compression via structured pruning and quantization. These approaches enable SLMs to retain much of the quality of their larger counterparts while significantly reducing training/serving costs and latency. We detail the impact of these techniques on a variety of use cases in a large professional social network platform and share deployment lessons, including hardware optimization strategies that improve speed and throughput for both predictive and reasoning-based applications in Recommendation Systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。