用图神经网络学习高效位置编码,提升图模型表达能力。
Learning Efficient Positional Encodings with Graph Neural Networks
- 基于消息传递GNN构建可学习的位置编码框架
- 线性复杂度下逼近等变函数,性能接近全量特征编码
- 适合大规模图数据的高效位置编码需求
位置编码(PEs)对图表示学习至关重要,因其能为本质上无位置感知的Transformer架构提供位置信息,并增强图神经网络(GNNs)的表达能力。然而,由于缺乏规范节点顺序且图规模大,设计强大且高效的图位置编码面临挑战。本文提出四个关键属性:稳定性、表达力、可扩展性和通用性。我们发现现有基于特征向量的方法难以同时满足这些要求。为此,提出PEARL框架,核心思想是利用消息传递GNN作为特征向量的非线性映射,生成强大而高效的PEs。关键挑战在于节点初始化需兼具表达力与置换等变性,我们通过随机输入或标准基向量初始化,并结合统计池化函数实现此目标。分析表明,PEARL以线性复杂度近似特征向量的等变函数,且严格证明其稳定性和高表达力。实验显示,PEARL在轻量级特征向量方法上表现更优,性能接近完整特征向量方法,但复杂度降低一到两个数量级。代码已公开于https://github.com/ehejin/Pearl-PE。
原文摘要 · Abstract (English)
Positional encodings (PEs) are essential for effective graph representation learning because they provide position awareness in inherently position-agnostic transformer architectures and increase the expressive capacity of Graph Neural Networks (GNNs). However, designing powerful and efficient PEs for graphs poses significant challenges due to the absence of canonical node ordering and the scale of the graph. {In this work, we identify four key properties that graph PEs should satisfy}: stability, expressive power, scalability, and genericness. We find that existing eigenvector-based PE methods often fall short of jointly satisfying these criteria. To address this gap, we introduce PEARL, a novel framework of learnable PEs for graphs. Our primary insight is that message-passing GNNs function as nonlinear mappings of eigenvectors, enabling the design of GNN architectures for generating powerful and efficient PEs. A crucial challenge lies in initializing node attributes in a manner that is both expressive and permutation equivariant. We tackle this by initializing GNNs with random node inputs or standard basis vectors, thereby unlocking the expressive power of message-passing operations, while employing statistical pooling functions to maintain permutation equivariance. Our analysis demonstrates that PEARL approximates equivariant functions of eigenvectors with linear complexity, while rigorously establishing its stability and high expressive power. Experimental evaluations show that PEARL outperforms lightweight versions of eigenvector-based PEs and achieves comparable performance to full eigenvector-based PEs, but with one or two orders of magnitude lower complexity. Our code is available at https://github.com/ehejin/Pearl-PE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。