高维嵌入未必更稳定,不同方法表现差异大
The Impact of Dimensionality on the Stability of Node Embeddings
- 系统测试5种方法在不同维度下的嵌入稳定性
- 部分方法(如node2vec)高维更稳定,部分则相反
- 稳定维度与最优预测性能不匹配,需权衡
以往研究发现,即使使用相同数据和超参数,节点嵌入方法在多次训练中仍会产生不同的表示和下游预测结果。然而,嵌入维度对这种不稳定性的影响尚不明确。本文系统分析了五种主流节点嵌入方法(ASNE、DGI、GraphSAGE、node2vec、VERSE)在不同维度下的稳定性,从表征和功能两个角度评估,覆盖多种数据集和重复训练。结果表明,维度对稳定性影响显著,但具体效应依赖于方法和稳定性定义。node2vec和ASNE在高维时趋于更稳定,而GraphSAGE和VERSE常呈现非单调行为或稳定性下降。此外,高稳定性对应的维度并不总是带来最佳下游性能。整体表明,嵌入维度对嵌入稳定性和下游预测有重要影响。
原文摘要 · Abstract (English)
Previous work has shown that node embedding methods can produce different representations and downstream predictions across repeated training runs, even when trained on the same data with identical hyperparameters. However, the role of embedding dimensionality in this instability remains poorly understood. In this work, we systematically analyze how embedding dimensionality affects the stability of embeddings from five widely used node embedding methods: ASNE, DGI, GraphSAGE, node2vec, and VERSE. We evaluate stability from both representational and functional perspectives across a broad range of dimensions, datasets, and repeated training runs, and relate the resulting stability patterns to predictive performance. Our results show that dimensionality can substantially affect embedding stability, although the observed effects depend strongly on the embedding method and stability notion considered. While node2vec and ASNE generally became more stable at higher dimensions, GraphSAGE and VERSE often exhibited non-monotonic behavior or decreasing stability. We further find that dimensions associated with high stability do not necessarily coincide with those yielding the strongest downstream performance. Overall, our findings demonstrate that embedding dimensionality can have a substantial impact on the stability of node embeddings and downstream predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。