解决多模态推荐中的可信度问题,提升模型在虚假内容下的鲁棒性。
Towards Trustworthy Multimodal Recommendation
- 通过软匹配机制动态修正不靠谱的视觉/文本特征,无需改动原有模型结构。
- 在不同污染程度下实验验证,该方法显著提升推荐系统抗干扰能力。
- 揭示了伪交互和伪边对模型鲁棒性的双重影响,适合做工业级推荐优化。
多模态推荐虽能融合视觉与文本信息提升效果,但在实际电商场景中,图像不一致或标题党等不可靠内容会引入虚假信号,导致现有推荐器在模态污染下表现脆弱。本文从方法与分析双角度推进可信多模态推荐:提出一种即插即用的模态级修正模块,利用轻量投影与基于Sinkhorn的软匹配,抑制不匹配的模态信号并保持语义一致性;同时发现,在噪声协同信号下,训练集伪交互的效用取决于先验信号对齐程度,传播图中的伪边也可能放大偏差。在多个数据集与基线模型上,不同污染水平下的实验证明该方法有效提升鲁棒性,并验证上述交互级观察。
原文摘要 · Abstract (English)
Recent advances in multimodal recommendation have demonstrated the effectiveness of incorporating visual and textual content into collaborative filtering. However, real-world deployments raise an increasingly important yet underexplored issue: trustworthiness. On modern e-commerce platforms, multimodal content can be misleading or unreliable (e.g., visually inconsistent product images or click-bait titles), injecting untrustworthy signals into multimodal representations and making existing recommenders brittle under modality corruption. In this work, we take a step towards trustworthy multimodal recommendation from both a method and an analysis perspective. First, we propose a plug-and-play modality-level rectification component that mitigates untrustworthy modality features by learning soft correspondences between items and multimodal features. Using lightweight projections and Sinkhorn-based soft matching, the rectification suppresses mismatched modality signals while preserving semantic consistency, and can be integrated into existing multimodal recommenders without architectural modifications. Second, we present two practical insights on interaction-level trustworthiness under noisy collaborative signals: (i) training-set pseudo interactions can help or hurt performance under noise depending on prior-signal alignment; and (ii) propagation-graph pseudo edges can also help or hurt robustness, as message passing may amplify misalignment. Extensive experiments on multiple datasets and backbones under varying corruption levels demonstrate improved robustness from modality rectification and validate the above interaction-level observations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。