arXiv:2601.01123cs.LGcs.AI2026-01

让GNN模型利用历史激活信息,提升深层网络的预测性能。

Learning from Historical Activations in Graph Neural Networks

  • 设计两阶段注意力机制,融合各层中间激活与图结构
  • 在多个基准上优于传统池化方法,深层模型提升更明显
  • 适合处理深度GNN中的过平滑问题,提升模型鲁棒性

图神经网络(GNN)在社交网络、分子化学等领域表现卓越。其关键组件是池化过程,将模型前向传播中计算出的节点特征聚合为下游任务使用的最终描述符。然而,现有池化方法仅使用最后一层的节点特征,忽略了前序层产生的关键中间激活(我们称之为历史图激活),这一缺陷在节点表示随层数显著变化或存在过平滑等问题时尤为严重。为此,我们提出HISTOGRAPH,一种基于注意力机制的两阶段聚合层:先对各层中间激活应用统一的层间注意力,再进行节点级注意力。该方法建模了节点表示在多层间的演化过程,同时结合节点历史激活与图结构,优化最终预测特征。在多个图分类基准上的实验表明,HISTOGRAPH性能显著优于传统方法,尤其在深层GNN中展现出更强的鲁棒性。

原文摘要 · Abstract (English)

Graph Neural Networks (GNNs) have demonstrated remarkable success in various domains such as social networks, molecular chemistry, and more. A crucial component of GNNs is the pooling procedure, in which the node features calculated by the model are combined to form an informative final descriptor to be used for the downstream task. However, previous graph pooling schemes rely on the last GNN layer features as an input to the pooling or classifier layers, potentially under-utilizing important activations of previous layers produced during the forward pass of the model, which we regard as historical graph activations. This gap is particularly pronounced in cases where a node's representation can shift significantly over the course of many graph neural layers, and worsened by graph-specific challenges such as over-smoothing in deep architectures. To bridge this gap, we introduce HISTOGRAPH, a novel two-stage attention-based final aggregation layer that first applies a unified layer-wise attention over intermediate activations, followed by node-wise attention. By modeling the evolution of node representations across layers, our HISTOGRAPH leverages both the activation history of nodes and the graph structure to refine features used for final prediction. Empirical results on multiple graph classification benchmarks demonstrate that HISTOGRAPH offers strong performance that consistently improves traditional techniques, with particularly strong robustness in deep GNNs.

图神经网络特征聚合注意力机制深层模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。