通过多视角增强与渐进融合，提升跨模态实体表示学习效果
More Perspectives, Stronger Signals: Multi-Perspective Enhancement and Progressive Fusion for Multimodal Entity Representation Learning
- 将每种模态拆分为互补视角，用解耦损失防止表征坍塌
- 动态校准跨模态交互，在稀疏条件下提升MRR 4.04%、Hits@1 11.17%
- 适合需要处理噪声多、数据稀疏的跨模态知识图谱任务
学习有效的多模态实体表示是实现多模态知识图谱补全（MMKGC）等推理任务的基础。然而，现有方法常面临模态内语义过平滑和跨模态噪声过滤无效的问题，尤其在数据稀疏或模糊条件下更为明显。为此，我们提出PrismF统一框架，通过多视角增强与渐进融合协同提升信号强度。PrismF利用多视角机制将每种模态分解为互补视图，并引入解耦损失约束，增强细粒度模态内语义；同时采用渐进融合策略，动态校准跨模态交互，使模型能突出有效信号并抑制噪声或不可靠信息。在三个公开基准上的实验表明，PrismF整体性能最强，其中在KVC16K数据集上相对提升4.04%（MRR）和11.17%（Hits@1）。代码已开源：https://github.com/HubuKG/PrismF。
原文摘要 · Abstract (English)
Learning effective multimodal entity representations is fundamental for reasoning tasks such as multimodal knowledge graph completion (MMKGC). However, existing methods often suffer from semantic over-smoothing within modalities and ineffective noise filtration across modalities, particularly under sparse or ambiguous conditions. To overcome these limitations, we propose PrismF, a unified framework that synergizes multi-perspective enhancement with progressive fusion to extract stronger signals from diverse inputs. PrismF enhances fine-grained intra-modal semantics through a multi-perspective mechanism that decomposes each modality into complementary views and constrains them with a decoupling loss to reduce representation collapse. Furthermore, it improves cross-modal integration through a progressive fusion strategy that dynamically calibrates inter-modal interactions, enabling the model to emphasize informative signals while suppressing noisy or unreliable ones. Extensive experiments on three public benchmarks show that PrismF achieves the strongest overall performance, including relative improvements of 4.04% in MRR and 11.17% in Hits@1 on KVC16K. Our code can be found at https://github.com/HubuKG/PrismF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。