融合多模态与协同过滤,提升大模型推荐精准度
Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation
- 用多模态大模型统一建模文本与非文本内容,生成综合物品表征
- 通过后置对齐机制融合内容与ID信息,显著提升用户偏好建模精度
- 适合做多模态推荐系统研发,尤其在数据多样场景下表现突出
序列推荐系统已从传统协同过滤发展至深度学习,近期更转向大语言模型(LLM)。然而,现有LLM主要依赖文本内容,忽视协同过滤信号及其它模态,导致推荐性能受限。为此,我们提出Molar:一种融合多模态内容与ID信息的多模态大语言模型序列推荐框架。Molar利用多模态大语言模型(MLLM)从文本与非文本数据中生成统一物品表征,实现全面的多模态建模并丰富物品嵌入。同时,通过后置对齐机制,将基于内容和基于ID的用户表征进行对齐,确保个性化精准性与强鲁棒性。通过结合多模态内容与协同过滤信号,Molar有效捕捉用户兴趣与上下文语义,显著优于传统及基于LLM的基线方法。大量实验证明其在序列推荐任务中的优越性。代码已公开于https://anonymous.4open.science/r/Molar-8B06/。
原文摘要 · Abstract (English)
Sequential recommendation (SR) systems have evolved significantly over the past decade, transitioning from traditional collaborative filtering to deep learning approaches and, more recently, to large language models (LLMs). While the adoption of LLMs has driven substantial advancements, these models inherently lack collaborative filtering information, relying primarily on textual content data neglecting other modalities and thus failing to achieve optimal recommendation performance. To address this limitation, we propose Molar, a Multimodal large language sequential recommendation framework that integrates multiple content modalities with ID information to capture collaborative signals effectively. Molar employs an MLLM to generate unified item representations from both textual and non-textual data, facilitating comprehensive multimodal modeling and enriching item embeddings. Additionally, it incorporates collaborative filtering signals through a post-alignment mechanism, which aligns user representations from content-based and ID-based models, ensuring precise personalization and robust performance. By seamlessly combining multimodal content with collaborative filtering insights, Molar captures both user interests and contextual semantics, leading to superior recommendation accuracy. Extensive experiments validate that Molar significantly outperforms traditional and LLM-based baselines, highlighting its strength in utilizing multimodal data and collaborative signals for sequential recommendation tasks. The source code is available at https://anonymous.4open.science/r/Molar-8B06/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。