LinkedIn用LLM实现语义搜索，效率提升75倍且效果更好。
Semantic Search At LinkedIn
- 用LLM判断相关性，结合嵌入检索和小模型蒸馏优化
- 在固定延迟下吞吐量提升75倍，近似教师模型的排序质量
- 适合需要高效语义搜索的工业级推荐系统
基于大语言模型（LLMs）的语义搜索可按语义而非关键词匹配进行检索，但规模化部署需突破推理效率瓶颈。本文介绍LinkedIn用于AI职位搜索与人脉搜索的LLM语义搜索框架，结合LLM相关性判别器、基于嵌入的检索，以及通过多教师蒸馏训练的紧凑小语言模型，联合优化相关性与用户参与度。该框架采用预填充导向的推理架构，协同模型剪枝、上下文压缩与文本嵌入混合交互，在固定延迟约束下使排名吞吐量提升超过75倍，同时保持接近教师模型水平的NDCG指标，成为首个在效率上媲美传统方法的生产级LLM排序系统，显著提升了质量和用户参与度。
原文摘要 · Abstract (English)
Semantic search with large language models (LLMs) enables retrieval by meaning rather than keyword overlap, but scaling it requires major inference efficiency advances. We present LinkedIn's LLM-based semantic search framework for AI Job Search and AI People Search, combining an LLM relevance judge, embedding-based retrieval, and a compact Small Language Model trained via multi-teacher distillation to jointly optimize relevance and engagement. A prefill-oriented inference architecture co-designed with model pruning, context compression, and text-embedding hybrid interactions boosts ranking throughput by over 75x under a fixed latency constraint while preserving near-teacher-level NDCG, enabling one of the first production LLM-based ranking systems with efficiency comparable to traditional approaches and delivering significant gains in quality and user engagement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。