提出MyGram模型，提升多模态知识图谱实体对齐精度。
MyGram: Modality-aware Graph Transformer with Global Distribution for Multi-modal Entity Alignment
- 设计模态扩散学习模块，捕捉模态内深层结构信息。
- 引入Gram Loss，使跨模态特征分布更一致，性能提升最高9.9%。
- 适合需要融合图文信息的实体对齐任务，尤其在噪声干扰下表现强。
多模态实体对齐旨在通过整合图像、文本等多源数据，丰富实体语义表示，识别两个多模态知识图谱中的等价实体。然而，现有方法常忽略各模态内部的结构上下文信息，易受浅层特征干扰。为此，本文提出MyGram：一种具有全局分布一致性的模态感知图变压器。我们设计了模态扩散学习模块，以捕获模态内的深层结构上下文，并实现细粒度的多模态融合。此外，引入Gram Loss作为正则化约束，通过最小化由多模态特征构成的四维平行体体积，实现跨模态全局分布一致性。我们在五个公开数据集上进行实验，结果表明，MyGram优于基线模型，在FBDB15K上Hits@1提升4.8%，FBYG15K上提升9.9%，DBP15K上提升4.3%。
原文摘要 · Abstract (English)
Multi-modal entity alignment aims to identify equivalent entities between two multi-modal Knowledge graphs by integrating multi-modal data, such as images and text, to enrich the semantic representations of entities. However, existing methods may overlook the structural contextual information within each modality, making them vulnerable to interference from shallow features. To address these challenges, we propose MyGram, a modality-aware graph transformer with global distribution for multi-modal entity alignment. Specifically, we develop a modality diffusion learning module to capture deep structural contextual information within modalities and enable fine-grained multi-modal fusion. In addition, we introduce a Gram Loss that acts as a regularization constraint by minimizing the volume of a 4-dimensional parallelotope formed by multi-modal features, thereby achieving global distribution consistency across modalities. We conduct experiments on five public datasets. Results show that MyGram outperforms baseline models, achieving a maximum improvement of 4.8% in Hits@1 on FBDB15K, 9.9% on FBYG15K, and 4.3% on DBP15K.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。