用贝叶斯更新机制防止知识遗忘,让知识图谱持续学习新信息。
Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding
- 将增量数据视为贝叶斯后验更新,自然实现持续学习。
- 在多个基准上超越现有方法,有效保留历史知识。
- 适合需要长期演化的知识图谱系统,如社交网络分析。
随着社交媒体和万维网成为信息传播的核心,高效组织与理解动态演变的网络内容至关重要。知识图谱(KG)为此提供了有力框架,但社交媒体中快速涌现的新话题、用户关系和事件使传统静态知识图谱嵌入(KGE)模型迅速过时。持续知识图谱嵌入(CKGE)旨在解决此问题,但现有方法普遍面临灾难性遗忘,即学习新知识(如新梗或热点事件)时丢失旧的但仍有价值的信息,导致无法有效捕捉数据演化。本文提出新型CKGE框架BAKE,将CKGE建模为序列贝叶斯推断问题,利用贝叶斯后验更新原则作为自然的持续学习策略。该原则对数据顺序不敏感,并提供理论保证以最大程度保留先验知识。具体而言,将每批新数据视为对模型先验的贝叶斯更新,通过维护后验分布,模型在多轮快照演化中仍能有效保留早期知识。此外,为约束跨快照的知识演化,引入持续聚类方法,通过正则化项保持实体嵌入的紧凑簇结构,确保语义一致性的同时允许受控适应新知识。在多个CKGE基准上的大量实验表明,BAKE在绝大多数情况下表现最佳。
原文摘要 · Abstract (English)
As social media and the World Wide Web become hubs for information dissemination, effectively organizing and understanding the vast amounts of dynamically evolving Web content is crucial. Knowledge graphs (KGs) provide a powerful framework for structuring this information. However, the rapid emergence of new hot topics, user relationships, and events in social media renders traditional static knowledge graph embedding (KGE) models rapidly outdated. Continual Knowledge Graph Embedding (CKGE) aims to address this issue, but existing methods commonly suffer from catastrophic forgetting, whereby older, but still valuable, information is lost when learning new knowledge (such as new memes or trending events). This means the model cannot effectively learn the evolution of the data. We propose a novel CKGE framework, BAKE. Unlike existing methods, BAKE formulates CKGE as a sequential Bayesian inference problem and utilizes the Bayesian posterior update principle as a natural continual learning strategy. This principle is insensitive to data order and provides theoretical guarantees to preserve prior knowledge as much as possible. Specifically, we treat each batch of new data as a Bayesian update to the model's prior. By maintaining the posterior distribution, the model effectively preserves earlier knowledge even as it evolves over multiple snapshots. Furthermore, to constrain the evolution of knowledge across snapshots, we introduce a continual clustering method that maintains the compact cluster structure of entity embeddings through a regularization term, ensuring semantic consistency while allowing controlled adaptation to new knowledge. We conduct extensive experiments on multiple CKGE benchmarks, which demonstrate that BAKE achieves the top performance in the vast majority of cases compared to existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。