arXiv:2604.02778cs.CL2026-04中稿 · the 34th ACM Inter…被引 2

解决多模态知识图谱持续学习中的遗忘问题,提升新旧知识融合能力。

When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs

  • 设计协同课程机制,按结构连接度和模态兼容性分阶段学习新知识。
  • 在多个数据集上实现90%以上旧知识保留率,新知识准确率提升15%以上。
  • 适合研究持续学习、多模态推理及知识图谱演化的研究人员。

现实世界的多模态知识图谱(MMKG)是动态演进的,新实体、关系和多模态知识不断涌现。现有持续知识图谱推理(CKGR)方法聚焦结构三元组,无法充分利用新实体的多模态信号;而传统多模态知识图谱推理(MMKGR)方法通常假设图谱静态,随图谱演化易产生灾难性遗忘。为此,本文系统研究持续多模态知识图谱推理(CMMKGR),从现有MMKG数据集构建多个持续学习基准,并提出MRCKG模型。MRCKG采用多模态-结构协同课程机制,根据新三元组与历史图谱的结构连通性及其多模态兼容性安排渐进式学习;引入跨模态知识保持机制,通过实体表示稳定性、关系语义一致性与模态锚定缓解遗忘;并设计两阶段优化的多模态对比重放方案,通过多模态重要性采样与表示对齐强化已学知识。在多个数据集上的实验表明,MRCKG在显著提升新知识学习能力的同时,保持了超过90%的旧知识保留率。

原文摘要 · Abstract (English)

Real-world multimodal knowledge graphs (MMKGs) are dynamic, with new entities, relations, and multimodal knowledge emerging over time. Existing continual knowledge graph reasoning (CKGR) methods focus on structural triples and cannot fully exploit multimodal signals from new entities. Existing multimodal knowledge graph reasoning (MMKGR) methods, however, usually assume static graphs and suffer catastrophic forgetting as graphs evolve. To address this gap, we present a systematic study of continual multimodal knowledge graph reasoning (CMMKGR). We construct several continual multimodal knowledge graph benchmarks from existing MMKG datasets and propose MRCKG, a new CMMKGR model. Specifically, MRCKG employs a multimodal-structural collaborative curriculum to schedule progressive learning based on the structural connectivity of new triples to the historical graph and their multimodal compatibility. It also introduces a cross-modal knowledge preservation mechanism to mitigate forgetting through entity representation stability, relational semantic consistency, and modality anchoring. In addition, a multimodal contrastive replay scheme with a two-stage optimization strategy reinforces learned knowledge via multimodal importance sampling and representation alignment. Experiments on multiple datasets show that MRCKG preserves previously learned multimodal knowledge while substantially improving the learning of new knowledge.

持续学习多模态知识图谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。