arXiv:2412.14978cs.IRcs.MM2024-12中稿 · ACM Web Search and…被引 91

通过频谱域融合抑制多模态噪声,提升推荐精度

Spectrum-based Modality Representation Fusion Graph Convolutional Network for Multimodal Recommendation

  • 将多模态特征转至频域,利用谱空间实现自适应噪声抑制
  • 在三个真实数据集上显著优于基线模型,提升推荐效果
  • 适合需要处理噪声多模态数据的推荐系统研究者

将多模态特征作为辅助信息融入推荐系统已成为趋势。现有方法多采用拼接、逐元素相加或注意力机制融合模态,但未考虑各模态内特有的噪声,导致跨模态噪声被放大。此外,模态间噪声特性差异使降噪与融合更复杂。为此,本文提出基于频谱的模态表示融合图卷积网络(SMORE),旨在同时捕捉单模态与融合偏好,并抑制模态噪声。SMORE将多模态特征投影至频域,利用谱空间进行融合;引入自适应滤波器,有效抑制各模态特有的动态噪声,同时保留通用模态模式。设计新型多模态图学习模块,挖掘物品间的潜在结构与语义关联。构建模态感知偏好模块,融合行为特征并平衡单模态与多模态信息,实现更精准的偏好建模。实验在三个真实数据集上验证了模型有效性,源代码已公开于https://github.com/kennethorq/SMORE。

原文摘要 · Abstract (English)

Incorporating multi-modal features as side information has recently become a trend in recommender systems. To elucidate user-item preferences, recent studies focus on fusing modalities via concatenation, element-wise sum, or attention mechanisms. Despite having notable success, existing approaches do not account for the modality-specific noise encapsulated within each modality. As a result, direct fusion of modalities will lead to the amplification of cross-modality noise. Moreover, the variation of noise that is unique within each modality results in noise alleviation and fusion being more challenging. In this work, we propose a new Spectrum-based Modality Representation (SMORE) fusion graph recommender that aims to capture both uni-modal and fusion preferences while simultaneously suppressing modality noise. Specifically, SMORE projects the multi-modal features into the frequency domain and leverages the spectral space for fusion. To reduce dynamic contamination that is unique to each modality, we introduce a filter to attenuate and suppress the modality noise adaptively while capturing the universal modality patterns effectively. Furthermore, we explore the item latent structures by designing a new multi-modal graph learning module to capture associative semantic correlations and universal fusion patterns among similar items. Finally, we formulate a new modality-aware preference module, which infuses behavioral features and balances the uni- and multi-modal features for precise preference modeling. This empowers SMORE with the ability to infer both user modality-specific and fusion preferences more accurately. Experiments on three real-world datasets show the efficacy of our proposed model. The source code for this work has been made publicly available at https://github.com/kennethorq/SMORE.

多模态推荐图神经网络噪声抑制频谱融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。