高绩效知识图谱嵌入模型预测不稳定，随机因素影响大。
Link Prediction or Perdition: the Seeds of Instability in Knowledge Graph Embeddings
- 系统分析多种模型在不同数据集上的预测稳定性
- 随机因素导致三元组预测差异大，嵌入空间变化显著
- 提升排名指标的配置未必更稳定，投票效果有限
知识图谱嵌入模型（KGEMs）是完成知识图谱的主要链接预测方法。标准评估协议侧重于基于排名的指标（如MRR或Hits@K），却常忽略随机种子对结果稳定性的影响。这些指标掩盖了个体预测和嵌入空间组织中的潜在不稳定性。本文对多种KGEMs在多个数据集上进行了系统的稳定性分析，发现高性能模型在三元组层面产生分歧性预测，且嵌入空间高度可变。通过隔离随机因素（初始化、三元组顺序、负采样、丢弃率、硬件），我们表明每个因素独立引发的不稳定性程度相当。此外，对于同一模型，提升MRR的超参数配置未必更稳定。尽管投票是已知的缓解机制，但仅能有限提升稳定性。这些发现揭示了当前基准测试协议的关键局限，对KGEMs在知识图谱补全中的可靠性提出质疑。
原文摘要 · Abstract (English)
Embedding models (KGEMs) constitute the main link prediction approach to complete knowledge graphs. Standard evaluation protocols emphasize rank-based metrics such as MRR or Hits@$K$, but usually overlook the influence of random seeds on result stability. Moreover, these metrics conceal potential instabilities in individual predictions and in the organization of embedding spaces. In this work, we conduct a systematic stability analysis of multiple KGEMs across several datasets. We find that high-performance models actually produce divergent predictions at the triple level and highly variable embedding spaces. By isolating stochastic factors (i.e., initialization, triple ordering, negative sampling, dropout, hardware), we show that each independently induces instability of comparable magnitude. Furthermore, for a given model, hyperparameter configurations with better MRR are not guaranteed to be more stable. Moreover, voting, albeit a known remediation mechanism, only provides a limited enhancement of stability. These findings highlight critical limitations of current benchmarking protocols, and raise concerns about the reliability of KGEMs for knowledge graph completion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。