用图检索增强补全缺失多模态信息,提升推荐系统鲁棒性。
Robust Multimodal Recommendation via Graph Retrieval-Enhanced Modality Completion

- 通过语义相关子图检索获取丰富上下文
- 联合编码查询节点与子图,完成缺失模态特征
- 适合处理数据不完整场景的推荐系统研究者
多模态数据在基于网络的推荐系统中至关重要,视觉与文本等多源信息可增强表征学习。然而,真实数据集常因传感器故障、标注稀缺或隐私限制导致模态缺失,严重降低模型性能。现有方法通常依赖查询节点自身或邻居信息重建缺失模态,但易忽略图中潜在的语义相关上下文。本文提出GRE-MC框架,引入模态感知的子图检索机制,从全图中选取语义相关的子图,为补全提供更丰富的上下文。随后,图变压器通过全局注意力联合编码查询节点与检索子图,完成缺失特征;可学习的稀疏路由码本将潜在嵌入正则化为紧凑基,提升鲁棒性。在多个多模态推荐基准上的实验表明,GRE-MC持续优于当前最优方法,验证了子图检索与联合编码图变压器的有效性。
原文摘要 · Abstract (English)
Multimodal data plays a critical role in web-based recommendation systems, where information from diverse modalities such as vision and text enhances representation learning. However, real-world multimodal datasets often suffer from modality incompleteness due to sensor failures, annotation scarcity, or privacy constraints, which substantially degrade model performance and reliability. One effective solution to address this issue is modality completion, which reconstructs missing features to provide modality-complete graphs for downstream tasks. Given a query node with missing multimodal features, existing modality completion methods typically infer information from the node itself or its neighbors to reconstruct the missing modality. However, these methods may overlook semantically relevant context in the graph, which contains valuable cues that are non-trivial to capture through simple methods like neighborhood aggregation. In this work, we propose GRE-MC, a Graph Retrieval-Enhanced Modality Completion framework, to overcome these limitations. By introducing a modality-aware subgraph retrieval mechanism, GRE-MC selects semantically relevant subgraphs from the entire graph, providing richer contextual information for completing missing modalities. Subsequently, a graph transformer jointly encodes the query node and the retrieved subgraph via global attention to complete the missing features, while a learnable sparse-routing codebook regularizes latent embeddings into compact bases for improved robustness. Extensive experiments on multimodal recommendation benchmarks demonstrate that GRE-MC consistently outperforms state-of-the-art methods, validating the effectiveness of subgraph retrieval and joint-encoding graph transformer for robust modality completion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。