直接从多模态特征学习物品表示,提升推荐效果
Learning Item Representations Directly from Multimodal Features for Effective Recommendation
- 不依赖物品ID嵌入,直接从多模态特征学习物品表示
- 在5个真实数据集上,NDCG@20平均提升4.21%
- 融合多模态特征并动态加权,适合多模态推荐场景
传统多模态推荐系统主要通过贝叶斯个性化排序(BPR)优化,将物品身份(ID)嵌入与多模态特征结合来学习物品表示。然而,我们的实证和理论分析表明,优化梯度存在显著偏向,倾向于从多模态特征中获取表示,而忽略物品ID嵌入。因此,即使多模态参数收敛,物品ID嵌入仍表现不佳。针对此问题,本文提出LIRDRec模型,直接从多模态特征中学习物品表示以提升推荐性能。考虑到各模态特征捕捉到的物品信息不同但相关,我们设计了融合模态特异性编码器的多模态转换机制,有效整合所有模态特征。此外,引入渐进式权重复制融合模块,逐步学习各模态在生成最终用户/物品表示中的权重。最后,利用多模态大语言模型(MLLMs)将物品图像转为文本,并通过大语言模型提取语义嵌入。在五个真实数据集上的实验验证了该方法的优越性。使用来自MLLMs和LLMs的嵌入时,相比原始嵌入,可使NDCG@20平均提升4.21%。
原文摘要 · Abstract (English)
Conventional multimodal recommender systems predominantly leverage Bayesian Personalized Ranking (BPR) optimization to learn item representations by amalgamating item identity (ID) embeddings with multimodal features. Nevertheless, our empirical and theoretical findings unequivocally demonstrate a pronounced optimization gradient bias in favor of acquiring representations from multimodal features over item ID embeddings. As a consequence, item ID embeddings frequently exhibit suboptimal characteristics despite the convergence of multimodal feature parameters. Given the rich informational content inherent in multimodal features, in this paper, we propose a novel model (i.e., LIRDRec) that learns item representations directly from these features to augment recommendation performance. Recognizing that features derived from each modality may capture disparate yet correlated aspects of items, we propose a multimodal transformation mechanism, integrated with modality-specific encoders, to effectively fuse features from all modalities. Moreover, to differentiate the influence of diverse modality types, we devise a progressive weight copying fusion module within LIRDRec. This module incrementally learns the weight assigned to each modality in synthesizing the final user or item representations. Finally, we utilize the powerful visual understanding of Multimodal Large Language Models (MLLMs) to convert the item images into texts and extract semantics embeddings upon the texts via LLMs. Empirical evaluations conducted on five real-world datasets validate the superiority of our approach relative to competing baselines. It is worth noting the proposed model, equipped with embeddings extracted from MLLMs and LLMs, can further improve the recommendation accuracy of NDCG@20 by an average of 4.21% compared to the original embeddings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。