用大模型提升推荐系统冷启动表现，且不增加计算负担
LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation
- 从大模型提取语义嵌入，仅选择性初始化推荐模型部分参数
- 在多个真实数据集上显著提升性能，计算开销几乎不变
- 适合工业级推荐系统，尤其冷启动场景下的应用
协同过滤（CF）广泛应用于工业推荐系统中以建模用户-物品交互，但在冷启动和数据稀疏场景下表现不佳。预训练大语言模型（LLM）蕴含丰富的语义知识，为解决该问题提供了新思路。然而，大规模部署LLM受限于其高计算开销和延迟。本文提出一种新颖且可扩展的LLM-RecSys框架LLMInit，通过选择性初始化策略将预训练LLM嵌入集成到CF模型中。我们发现当CF模型规模扩大并匹配大模型嵌入维度时会出现嵌入坍缩问题，并通过引入高效的采样方法（随机、均匀、方差基于）有效规避。在多个真实数据集上的全面实验表明，LLMInit显著提升推荐性能的同时保持低计算成本，为工业应用提供了一种实用且可扩展的解决方案。为促进产业落地与未来研究，代码已开源：https://github.com/DavidZWZ/LLMInit。
原文摘要 · Abstract (English)
Collaborative filtering (CF) is widely adopted in industrial recommender systems (RecSys) for modeling user-item interactions across numerous applications, but often struggles with cold-start and data-sparse scenarios. Recent advancements in pre-trained large language models (LLMs) with rich semantic knowledge, offer promising solutions to these challenges. However, deploying LLMs at scale is hindered by their significant computational demands and latency. In this paper, we propose a novel and scalable LLM-RecSys framework, LLMInit, designed to integrate pretrained LLM embeddings into CF models through selective initialization strategies. Specifically, we identify the embedding collapse issue observed when CF models scale and match the large embedding sizes in LLMs and avoid the problem by introducing efficient sampling methods, including, random, uniform, and variance-based selections. Comprehensive experiments conducted on multiple real-world datasets demonstrate that LLMInit significantly improves recommendation performance while maintaining low computational costs, offering a practical and scalable solution for industrial applications. To facilitate industry adoption and promote future research, we provide open-source access to our implementation at https://github.com/DavidZWZ/LLMInit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。