arXiv:2409.12740cs.IRcs.AI2024-09被引 130

用分层大模型提升推荐系统,从商品描述中提取特征并预测用户兴趣。

HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling

论文配图:HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling
图 1 · 摘自论文原文
  • 分两层:先用物品LLM提取商品文本特征,再用用户LLM建模兴趣
  • 在两个数据集上超越传统模型,7B参数配置表现最佳
  • 无需复杂调参,训练和部署高效,适合真实场景落地

大型语言模型(LLMs)在多个领域取得显著成果,促使研究者探索其在推荐系统中的潜力。然而,现有方法仅带来有限性能提升。三个关键问题仍未充分探讨:第一,预训练权重蕴含的通用知识实际价值;第二,推荐任务是否需要微调;第三,LLMs在推荐系统中能否像其他领域一样展现可扩展性。本文提出一种新型分层大语言模型(HLLM)架构,用于增强序列推荐系统。该方法采用双层设计:第一层物品LLM从商品详细文本描述中提取丰富内容特征,第二层用户LLM基于用户交互历史,利用这些特征预测其未来兴趣。大量实验表明,该方法有效利用开源LLMs的预训练能力,进一步微调可带来显著性能提升。此外,HLLM具备优异可扩展性,最大配置使用7B参数进行物品特征提取与用户兴趣建模。同时,其训练与推理效率高,适用于实际应用。在PixelRec和Amazon Reviews两个大规模数据集上的评估显示,HLLM达到当前最优效果,大幅超越传统基于ID的模型。在线A/B测试中,HLLM展现出明显收益,验证了其在真实推荐场景中的实际影响力。代码已公开于https://github.com/bytedance/HLLM。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have achieved remarkable success in various fields, prompting several studies to explore their potential in recommendation systems. However, these attempts have so far resulted in only modest improvements over traditional recommendation models. Moreover, three critical questions remain under-explored: firstly, the real value of LLMs' pre-trained weights, often considered to encapsulate world knowledge; secondly, the necessity of fine-tuning for recommendation tasks; lastly, whether LLMs can exhibit the same scalability benefits in recommendation systems as they do in other domains. In this paper, we propose a novel Hierarchical Large Language Model (HLLM) architecture designed to enhance sequential recommendation systems. Our approach employs a two-tier model: the first Item LLM extracts rich content features from the detailed text description of the item, while the second User LLM utilizes these features to predict users' future interests based on their interaction history. Extensive experiments demonstrate that our method effectively leverages the pre-trained capabilities of open-source LLMs, and further fine-tuning leads to significant performance boosts. Additionally, HLLM achieves excellent scalability, with the largest configuration utilizing 7B parameters for both item feature extraction and user interest modeling. Moreover, HLLM offers excellent training and serving efficiency, making it practical in real-world applications. Evaluations on two large-scale datasets, PixelRec and Amazon Reviews, show that HLLM achieves state-of-the-art results, outperforming traditional ID-based models by a wide margin. In online A/B testing, HLLM showcases notable gains, validating its practical impact in real-world recommendation scenarios. Codes are available at https://github.com/bytedance/HLLM.

推荐系统大模型序列推荐分层模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。