压缩小模型+优化部署，让搜索系统快10倍且不降质
Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search
- 用剪枝技术缩小模型40%，保持精度不变
- 输入上下文压缩10倍，准确率几乎无损
- 在百万级请求/秒下部署，吞吐量提升10倍
大型语言模型在相关性排序和语义搜索等任务中表现出色，但其部署成本过高，难以满足工业应用对延迟和吞吐量的严苛要求。本文分享了在 LinkedIn 构建纯文本解码器仅有的小型语言模型（SLM）用于语义搜索的经验与效率洞察。我们采用剪枝等模型压缩技术，将模型规模减少高达40%而保持准确率；提出上下文压缩技术，使输入长度缩短10倍，仅造成轻微精度损失；同时总结了在GPU上规模化部署该系统的实用经验，最终实现真实部署中系统吞吐量提升10倍，且满足质量标准。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated impressive quality when applied to predictive tasks such as relevance ranking and semantic search. However, deployment of such LLMs remains prohibitively expensive for industry applications with strict latency and throughput requirements. In this work, we present lessons and efficiency insights from developing a purely text-based decoder-only Small Language Model (SLM) for a semantic search application at LinkedIn. Particularly, we discuss model compression techniques such as pruning that allow us to reduce the model size by up to $40\%$ while maintaining the accuracy. Additionally, we present context compression techniques that allow us to reduce the input context length by up to $10$x with minimal loss of accuracy. Finally, we present practical lessons from optimizing the serving infrastructure for deploying such a system on GPUs at scale, serving millions of requests per second. Taken together, this allows us to increase our system's throughput by $10$x in a real-world deployment, while meeting our quality bar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。