解决推荐中模态缺失问题,让系统能自动补全丢失的图文信息。
Disentangling and Generating Modalities for Recommendation in Missing Modality Scenarios
- 将模态特征拆分为通用和特有部分,提升表示能力。
- 通过其他模态生成缺失特征,在多种缺失场景下表现更优。
- 支持跨模态检索,适合真实应用中的不完整数据场景。
多模态推荐系统(MRS)通过融合图像、文本、音频等多源信息显著提升了个性化推荐效果。然而,现有方法对模态缺失场景关注不足,且忽视了模态特征的独特性,导致在实际应用中性能大幅下降。为此,我们提出针对缺失模态场景的解耦与生成推荐框架DGMRec。该框架从信息论角度将模态特征解耦为通用特征与特定模态特征,增强表示能力;并基于其他模态对齐特征与用户模态偏好,生成缺失模态特征。大量实验表明,DGMRec在多种挑战性场景(包括模态缺失、新物品设置、不同缺失率及缺失类型)下均优于当前先进MRS。此外,其生成式设计支持跨模态检索,这是现有MRS无法实现的任务,展现出更强适应性与现实应用潜力。代码已开源。
原文摘要 · Abstract (English)
Multi-modal recommender systems (MRSs) have achieved notable success in improving personalization by leveraging diverse modalities such as images, text, and audio. However, two key challenges remain insufficiently addressed: (1) Insufficient consideration of missing modality scenarios and (2) the overlooking of unique characteristics of modality features. These challenges result in significant performance degradation in realistic situations where modalities are missing. To address these issues, we propose Disentangling and Generating Modality Recommender (DGMRec), a novel framework tailored for missing modality scenarios. DGMRec disentangles modality features into general and specific modality features from an information-based perspective, enabling richer representations for recommendation. Building on this, it generates missing modality features by integrating aligned features from other modalities and leveraging user modality preferences. Extensive experiments show that DGMRec consistently outperforms state-of-the-art MRSs in challenging scenarios, including missing modalities and new item settings as well as diverse missing ratios and varying levels of missing modalities. Moreover, DGMRec's generation-based approach enables cross-modal retrieval, a task inapplicable for existing MRSs, highlighting its adaptability and potential for real-world applications. Our code is available at https://github.com/ptkjw1997/DGMRec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。