用大模型提升推荐系统,解决冷启动和性能瓶颈问题。
Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Models
- 分三步:预训练、特征对齐、解耦专家模型
- 在三个数据集上显著提升冷门商品推荐效果
- 适合想用大模型改进推荐系统的研究人员
序列推荐(SR)旨在利用用户历史交互的顺序模式精准捕捉偏好。然而,现有方法高度依赖协同数据,导致冷启动问题和性能不足。尽管大语言模型(LLM)表现优异,但在实际推荐系统中受限于推理延迟高、分布统计不完整及灾难性遗忘等问题。为此,我们提出预训练、对齐与解耦(PAD)框架,以增强基于LLM的SR模型。首先分别预训练SR与LLM模型,获得协同与文本嵌入;接着引入基于高斯核多核最大均值差异的特征推荐锚定对齐损失;最后采用三专家架构,包含对齐专家与模态特定专家,并通过频率感知方式微调解耦嵌入。在三个公开数据集上的实验验证了PAD的有效性,显著提升性能,且兼容多种SR主干模型,尤其对冷门物品效果突出。代码与数据集可在https://github.com/Applied-Machine-Learning-Lab/PAD获取。
原文摘要 · Abstract (English)
Sequential Recommendation (SR) aims to leverage the sequential patterns in users' historical interactions to accurately track their preferences. However, the primary reliance of existing SR methods on collaborative data results in challenges such as the cold-start problem and sub-optimal performance. Concurrently, despite the proven effectiveness of large language models (LLMs), their integration into commercial recommender systems is impeded by issues such as high inference latency, incomplete capture of all distribution statistics, and catastrophic forgetting. To address these issues, we introduce a novel Pre-train, Align, and Disentangle (PAD) framework to enhance SR models with LLMs. In particular, we initially pre-train both the SR and LLM models to obtain collaborative and textual embeddings. Subsequently, we propose a characteristic recommendation-anchored alignment loss using multi-kernel maximum mean discrepancy with Gaussian kernels. Lastly, a triple-experts architecture, comprising aligned and modality-specific experts with disentangled embeddings, is fine-tuned in a frequency-aware manner. Experimental results on three public datasets validate the efficacy of PAD, indicating substantial enhancements and compatibility with various SR backbone models, particularly for cold items. The code and datasets are accessible for reproduction at https://github.com/Applied-Machine-Learning-Lab/PAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。