arXiv:2410.06742cs.LG2024-10

让知识图谱推理支持未见实体关系,突破传统模型局限。

Inference over Unseen Entities, Relations and Literals on Knowledge Graphs

  • 用子词编码构建三元组嵌入,实现跨实体关系的特征复用。
  • 在语义有意义的符号表示数据集上,提升4种模型的链接预测性能。
  • 适合动态知识图谱、新实体关系推理场景,开源可复现。

近年来,知识图谱嵌入模型在归纳设置下成功应用于链接预测、查询回答等任务。然而,传统归纳设置无法处理未见实体、关系甚至数值或非数值字面量。尽管已有研究探索归纳场景,但对未见实体、关系和字面量的推理仍无进展。这限制了模型应对包含异构信息的真实世界动态知识图谱的能力。为此,本文提出一种改进方法:引入注意力式字节对编码层(BytE),通过字节对编码的子词单元序列构建三元组嵌入。相比传统方法,BytE通过权重共享实现大规模特征复用,使模型学习子词单元嵌入而非直接学习实体与关系嵌入,从而摆脱嵌入矩阵大小对知识图谱唯一实体与关系数量的依赖。实验表明,BytE在三元组符号表示具有语义意义的数据集上,显著提升了4种知识图谱嵌入模型的链接预测性能;但在使用纯数字或URI表示实体与关系的知识图谱上,其优势消失。作者已开源BytE实现,以促进可复现研究。

原文摘要 · Abstract (English)

In recent years, knowledge graph embedding models have been successfully applied in the transductive setting to tackle various challenging tasks including link prediction, and query answering. Yet, the transductive setting does not allow for reasoning over unseen entities, relations, let alone numerical or non-numerical literals. Although increasing efforts are put into exploring inductive scenarios, inference over unseen entities, relations, and literals has yet to come. This limitation prohibits the existing methods from handling real-world dynamic knowledge graphs involving heterogeneous information about the world. Here, we propose a remedy to this limitation. We propose the attentive byte-pair encoding layer (BytE) to construct a triple embedding from a sequence of byte-pair encoded subword units of entities and relations. Compared to the conventional setting, BytE leads to massive feature reuse via weight tying, since it forces a knowledge graph embedding model to learn embeddings for subword units instead of entities and relations directly. Consequently, the size of the embedding matrices are not anymore bound to the unique number of entities and relations of a knowledge graph. Experimental results show that BytE improves the link prediction performance of 4 knowledge graph embedding models on datasets where the syntactic representations of triples are semantically meaningful. However, benefits of training a knowledge graph embedding model with BytE dissipate on knowledge graphs where entities and relations are represented with plain numbers or URIs. We provide an open source implementation of BytE to foster reproducible research.

知识图谱归纳推理子词编码动态图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。