用大模型生成推荐结果，提升视频推荐效果
PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations
- 用语义编码替代物品名，让大模型理解推荐内容
- 在真实数据上训练后，推荐准确率显著超越传统系统
- 适合想用大模型做个性化推荐的工业级应用
大型语言模型（LLMs）为信息任务带来了新的建模与计算范式。推荐系统作为关键应用领域，有望从这些模型的序列建模能力与世界知识中获益。本文提出PLUM框架，用于将预训练大模型适配至工业级推荐任务。该框架包括：使用语义ID进行物品分词、在领域数据上进行持续预训练（CPT），以及针对推荐目标的任务微调。微调阶段聚焦生成式召回，直接训练模型根据用户上下文生成推荐物品的语义ID。我们在大规模内部视频推荐数据集上进行了全面实验。结果表明，相比依赖大嵌入表的优化生产模型，PLUM在召回性能上取得显著提升。我们还开展了模型扩展性研究，总结了关于CPT的经验，改进了语义ID设计，并介绍了支持向数十亿用户部署的训练与推理方法。
原文摘要 · Abstract (English)
Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world knowledge inherent in these large models. In this paper, we introduce PLUM, a framework designed to adapt pre-trained LLMs for industry-scale recommendation tasks. PLUM consists of item tokenization using Semantic IDs, continued pre-training (CPT) on domain-specific data, and task-specific fine-tuning for recommendation objectives. For fine-tuning, we focus particularly on generative retrieval, where the model is directly trained to generate Semantic IDs of recommended items based on user context. We conduct comprehensive experiments on large-scale internal video recommendation datasets. Our results demonstrate that PLUM achieves substantial improvements for retrieval compared to a heavily-optimized production model built with large embedding tables. We also present a scaling study for the model's retrieval performance, our learnings about CPT, a few enhancements to Semantic IDs, along with an overview of the training and inference methods that enable launching this framework to billions of users in YouTube.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。