用扩散模型学习图表示,效果媲美顶尖方法。
Graffe: Graph Representation Learning via Diffusion Probabilistic Models
- 通过编码器压缩图结构,再用其指导去噪过程。
- 在11个数据集上9个达最优,线性探测表现优秀。
- 理论证明去噪目标能最大化数据与表示的互信息。
扩散概率模型(DPMs)虽在生成高质量样本方面表现优异,但在表示学习领域仍鲜受关注。尽管近期研究揭示其捕捉视觉语义的潜力,但将其应用于图表示学习仍处于初级阶段。本文提出Graffe,一种用于图表示学习的自监督扩散模型。其图编码器将源图压缩为紧凑表示,该表示作为条件引导扩散解码器的去噪过程。我们首先探讨了将扩散模型应用于表示学习的理论基础,证明去噪目标隐式最大化数据与其表示之间的条件互信息。具体而言,去噪评分匹配损失的负对数是条件互信息的可计算下界。通过一系列案例研究验证了理论见解。实验表明,Graffe在节点分类和图分类任务的线性探测设置下表现竞争力,在11个真实世界数据集中的9个达到领先水平。结果表明,强大的生成模型,尤其是扩散模型,可成为图表示学习的有效工具。
原文摘要 · Abstract (English)
Diffusion probabilistic models (DPMs), widely recognized for their potential to generate high-quality samples, tend to go unnoticed in representation learning. While recent progress has highlighted their potential for capturing visual semantics, adapting DPMs to graph representation learning remains in its infancy. In this paper, we introduce Graffe, a self-supervised diffusion model proposed for graph representation learning. It features a graph encoder that distills a source graph into a compact representation, which, in turn, serves as the condition to guide the denoising process of the diffusion decoder. To evaluate the effectiveness of our model, we first explore the theoretical foundations of applying diffusion models to representation learning, proving that the denoising objective implicitly maximizes the conditional mutual information between data and its representation. Specifically, we prove that the negative logarithm of the denoising score matching loss is a tractable lower bound for the conditional mutual information. Empirically, we conduct a series of case studies to validate our theoretical insights. In addition, Graffe delivers competitive results under the linear probing setting on node and graph classification tasks, achieving state-of-the-art performance on 9 of the 11 real-world datasets. These findings indicate that powerful generative models, especially diffusion models, serve as an effective tool for graph representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。