利用模态互补性提升多模态知识图谱补全效果
Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completion
- 通过模态互补融合机制整合多视角信息
- 在五个数据集上达到领先性能，显著优于现有方法
- 适合研究多模态表示学习与知识图谱的学者
多模态知识图谱补全（MMKGC）旨在利用实体的多模态和结构信息挖掘隐藏的世界知识。然而，多模态知识图谱中模态分布不均的问题，导致难以有效利用额外模态数据进行鲁棒的实体表示。现有方法多依赖注意力或门控融合机制，忽略了多模态数据中的互补性。本文提出新型框架Mixture of Complementary Modality Experts（MoCME），包含互补引导模态知识融合（CMKF）模块和熵引导负采样（EGNS）机制。CMKF模块利用模态内与模态间互补性融合多视图、多模态嵌入，增强实体表示；EGNS机制动态优先选择信息量大且不确定的负样本，提升训练效率与模型鲁棒性。在五个基准数据集上的大量实验表明，MoCME性能显著超越现有方法，达到当前最优水平。
原文摘要 · Abstract (English)
Multi-modal Knowledge Graph Completion (MMKGC) aims to uncover hidden world knowledge in multimodal knowledge graphs by leveraging both multimodal and structural entity information. However, the inherent imbalance in multimodal knowledge graphs, where modality distributions vary across entities, poses challenges in utilizing additional modality data for robust entity representation. Existing MMKGC methods typically rely on attention or gate-based fusion mechanisms but overlook complementarity contained in multi-modal data. In this paper, we propose a novel framework named Mixture of Complementary Modality Experts (MoCME), which consists of a Complementarity-guided Modality Knowledge Fusion (CMKF) module and an Entropy-guided Negative Sampling (EGNS) mechanism. The CMKF module exploits both intra-modal and inter-modal complementarity to fuse multi-view and multi-modal embeddings, enhancing representations of entities. Additionally, we introduce an Entropy-guided Negative Sampling mechanism to dynamically prioritize informative and uncertain negative samples to enhance training effectiveness and model robustness. Extensive experiments on five benchmark datasets demonstrate that our MoCME achieves state-of-the-art performance, surpassing existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。