提出SEPAL算法，在大规模知识图谱上实现高效高质量嵌入。
Scalable Feature Learning on Huge Knowledge Graphs for Downstream Machine Learning
- 仅在核心实体上优化嵌入，通过消息传递传播至全图。
- 在7个大型图谱、46项任务上显著超越现有方法。
- 可在普通硬件上处理超大规模知识图谱，适合工业级应用。
许多机器学习任务可从外部知识中获益。大型知识图谱存储此类知识，嵌入方法可将其提炼为下游应用的向量表示。然而当前模型存在两大局限：主要针对链接预测优化，采用局部对比学习；且在最大规模图谱上应用时需大量工程投入，受限于GPU内存。为此，我们提出SEPAL：一种专为大规模知识图谱设计的可扩展嵌入传播算法，旨在大规模生成适用于下游任务的高质量嵌入。SEPAL的核心思想是仅在少量核心实体上优化嵌入，再通过消息传递将嵌入传播至全图，以保证全局一致性。我们在7个大规模知识图谱和46项下游机器学习任务上评估SEPAL，结果表明其在下游任务上显著优于先前方法。此外，SEPAL可扩展基础嵌入模型，使超大规模知识图谱能在普通硬件上完成训练。
原文摘要 · Abstract (English)
Many machine learning tasks can benefit from external knowledge. Large knowledge graphs store such knowledge, and embedding methods can be used to distill it into ready-to-use vector representations for downstream applications. For this purpose, current models have however two limitations: they are primarily optimized for link prediction, via local contrastive learning, and their application to the largest graphs requires significant engineering effort due to GPU memory limits. To address these, we introduce SEPAL: a Scalable Embedding Propagation ALgorithm for large knowledge graphs designed to produce high-quality embeddings for downstream tasks at scale. The key idea of SEPAL is to ensure global embedding consistency by optimizing embeddings only on a small core of entities, and then propagating them to the rest of the graph with message passing. We evaluate SEPAL on 7 large-scale knowledge graphs and 46 downstream machine learning tasks. Our results show that SEPAL significantly outperforms previous methods on downstream tasks. In addition, SEPAL scales up its base embedding model, enabling fitting huge knowledge graphs on commodity hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。