arXiv:2412.17734cs.LGstat.ML2024-12

用可学习的神经网络实现图谱嵌入,提升精度与鲁棒性。

LASE: Learned Adjacency Spectral Embeddings

  • 将梯度下降迭代转化为GNN层,构建可训练的谱嵌入模型
  • 在链接预测和节点分类任务中超越传统GNN与谱方法
  • 适合需要高效、可解释谱嵌入的图学习场景

我们提出一种基于梯度下降与算法展开原理的神经架构,用于从图输入中学习节点邻接谱嵌入(ASE)。将每轮梯度下降迭代重新解释为一个图神经网络(GNN)层,通过训练逼近真实谱嵌入。由此得到的参数化模型称为学习型邻接谱嵌入(LASE),具备可解释性、参数高效、对未观测边鲁棒,且推理复杂度可控。LASE层融合图卷积网络(GCN)与全连接图注意力网络(GAT)模块,因仅靠GCN局部聚合无法表达所需图特征向量。我们引入稀疏注意力与分层参数解耦等改进,实现近似误差与计算开销的良好平衡,甚至优于科学计算库中的优化特征分解方法。由于LASE对参数与图输入均是可微的,可无缝集成至(半)监督图表示学习框架中,端到端训练出具有判别性的谱嵌入,在链接预测与节点分类任务上表现优异,即便对比使用任务无关预计算谱位置编码的GNN也更优。

原文摘要 · Abstract (English)

We put forth a principled design of a neural architecture to learn nodal Adjacency Spectral Embeddings (ASE) from graph inputs. By bringing to bear the gradient descent (GD) method and leveraging the principle of algorithm unrolling, we truncate and re-interpret each GD iteration as a layer in a graph neural network (GNN) that is trained to approximate the ASE. Accordingly, we call the resulting embeddings and our parametric model Learned ASE (LASE), which is interpretable, parameter efficient, robust to inputs with unobserved edges, and offers controllable complexity during inference. LASE layers combine Graph Convolutional Network (GCN) and fully-connected Graph Attention Network (GAT) modules, which is intuitively pleasing since GCN-based local aggregations alone are insufficient to express the sought graph eigenvectors. We propose several refinements to the unrolled LASE architecture (such as sparse attention in the GAT module and decoupled layerwise parameters) that offer favorable approximation error versus computation tradeoffs; even outperforming heavily-optimized eigendecomposition routines from scientific computing libraries. Because LASE is a differentiable function with respect to its parameters as well as its graph input, we can seamlessly integrate it as a trainable module within a larger (semi-)supervised graph representation learning pipeline. The resulting end-to-end system effectively learns ``discriminative ASEs'' that exhibit competitive performance in supervised link prediction and node classification tasks, outperforming a GNN even when the latter is endowed with open loop, meaning task-agnostic, precomputed spectral positional encodings.

图神经网络谱嵌入可学习模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。