通过预测节点潜在表示,避免依赖输入重建和人工增强。
NodeJEPA: Structure-Conditioned Latent Prediction for Node-Level Graph Self-Supervised Learning

- 用结构感知的邻域掩码训练编码器预测隐藏表示。
- 在标准数据集上线性探测准确率最高达94.6%。
- 适合需要稳定结构学习的图神经网络研究者。
图上的自监督学习主要依赖于需精心设计增强的对比方法,以及在输入空间中重构节点属性的生成方法。这两种范式可能将表示与低层输入统计纠缠,而非关系结构。联合嵌入预测架构(JEPA)则通过预测潜在目标而非重建输入来学习。近期工作已探索该思想在图级表征学习中的应用,但如何为节点级任务设计类似JEPA的目标,以及预测器应基于何种结构信号进行条件化,仍不明确。本文提出NodeJEPA,一种面向节点级图自监督学习的联合嵌入预测架构。NodeJEPA对结构感知的k跳自子图进行掩码,并训练上下文编码器预测被掩码节点的潜在表示。这些目标来自带有停止梯度的EMA更新目标编码器。一个结构条件化预测器通过交叉注意力整合谱特征与中心性描述符。方差、协方差及拉普拉斯谱正则化有助于稳定嵌入几何结构,可选课程学习在训练过程中逐步增加掩码难度。由于预测发生在潜在空间,NodeJEPA不依赖输入重建或手工设计的图增强。我们在标准节点分类基准上评估了NodeJEPA在线性探测与微调协议下的表现,并对掩码、预测及正则化设计进行了消融实验。本研究为图上的节点级JEPA式潜在预测提供了实用方案,并明确了结构条件化在表征学习中的作用。代码、配置和评估脚本已在https://github.com/OliverZ-dot/Node-Jepa公开。
原文摘要 · Abstract (English)
Self-supervised learning on graphs is largely shaped by contrastive methods that depend on carefully designed augmentations, and by generative methods that reconstruct node attributes in the input space. Both paradigms can entangle representations with low-level input statistics rather than with relational structure. Joint-embedding predictive architectures (JEPA) instead learn by predicting latent targets rather than reconstructing inputs. Recent work has explored this idea for graph-level representation learning, but how to design JEPA-style objectives for node-level tasks, and which structural signals the predictor should condition on, remains less clear. We present NodeJEPA, a joint-embedding predictive architecture for node-level graph self-supervised learning. NodeJEPA masks structure-aware k-hop ego-subgraphs and trains a context encoder to predict the latent representations of the masked nodes. These targets come from an EMA-updated target encoder with stop-gradient. A structure-conditioned predictor integrates spectral and centrality descriptors through cross-attention. Variance, covariance, and Laplacian spectral regularizers help stabilize the embedding geometry, and an optional curriculum gradually increases masking difficulty during training. Because prediction occurs in latent space, NodeJEPA does not rely on input reconstruction or hand-crafted graph augmentations. We evaluate NodeJEPA on standard node classification benchmarks under linear probing and fine-tuning protocols, and conduct ablations on masking, prediction, and regularization design choices. Our study offers a practical recipe for node-level JEPA-style latent prediction on graphs, and clarifies when structural conditioning helps representation learning. Code, configurations, and evaluation scripts are publicly available at https://github.com/OliverZ-dot/Node-Jepa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。