用双视角融合大模型与原始文本,提升推荐系统准确性。
DMESR: Dual-view MLLM-based Enhancing Framework for Multimodal Sequential Recommendation
- 通过对比学习对齐多模态语义表示,解决跨模态不一致问题。
- 引入交叉注意力融合模块,保留原始文本的细粒度语义信息。
- 在三个真实数据集上验证,适合作为多模态推荐系统改进方案。
序列推荐系统(SRS)旨在基于用户历史行为预测其下一步交互,但仍面临数据稀疏性挑战。随着多模态大语言模型(MLLM)的快速发展,利用其多模态理解能力丰富物品语义表征已成为增强SRS的有效策略。然而,现有基于MLLM的推荐方法仍存在两大局限:一是难以有效对齐多模态表示,导致跨模态语义信息利用不充分;二是过度依赖MLLM生成内容,忽视了物品原始文本中的细粒度语义线索。为此,本文提出一种双视角MLLM增强框架DMESR。针对表示不对齐问题,采用对比学习机制对齐由MLLM生成的跨模态语义表示;针对细粒度语义丢失问题,设计交叉注意力融合模块,将粗粒度的MLLM语义与原始文本的细粒度语义进行融合。最终,这两种融合表示可无缝集成至下游序列推荐模型中。在三个真实世界数据集及三种主流序列推荐架构上的实验表明,所提方法具有卓越的性能和良好的泛化能力。
原文摘要 · Abstract (English)
Sequential Recommender Systems (SRS) aim to predict users' next interaction based on their historical behaviors, while still facing the challenge of data sparsity. With the rapid advancement of Multimodal Large Language Models (MLLMs), leveraging their multimodal understanding capabilities to enrich item semantic representation has emerged as an effective enhancement strategy for SRS. However, existing MLLM-enhanced recommendation methods still suffer from two key limitations. First, they struggle to effectively align multimodal representations, leading to suboptimal utilization of semantic information across modalities. Second, they often overly rely on MLLM-generated content while overlooking the fine-grained semantic cues contained in the original textual data of items. To address these issues, we propose a Dual-view MLLM-based Enhancing framework for multimodal Sequential Recommendation (DMESR). For the misalignment issue, we employ a contrastive learning mechanism to align the cross-modal semantic representations generated by MLLMs. For the loss of fine-grained semantics, we introduce a cross-attention fusion module that integrates the coarse-grained semantic knowledge obtained from MLLMs with the fine-grained original textual semantics. Finally, these two fused representations can be seamlessly integrated into the downstream sequential recommendation models. Extensive experiments conducted on three real-world datasets and three popular sequential recommendation architectures demonstrate the superior effectiveness and generalizability of our proposed approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。