arXiv:2506.17576cs.LG2025-06

提出渐进式训练法,解决深度图卷积网络过平滑问题。

Towards a deeper GCN: Alleviate over-smoothing with iterative training and fine-tuning

  • 分层渐进训练,逐步构建深层图网络
  • 32层模型准确率显著提升,超越现有方法
  • 适配多种图神经网络,无需修改结构

图卷积网络(GCN)在深度架构中因过平滑导致性能严重下降。我们实证分析发现,可训练线性变换是加剧特征坍塌的关键因素,即使在8层中也显著影响表现;而移除这些变换的Simplified GCN(SGC)可在32层内保持特征多样性。因此,我们提出分层渐进训练(LGT),通过三重机制:(1)逐层训练稳定优化,(2)低秩适应微调浅层加速训练,(3)恒等初始化促进新层融合与收敛。在基准数据集上的实验表明,LGT在原生GCN上实现最先进性能,32层设置下准确率显著提升。作为通用训练策略,可无缝结合PairNorm、ContraNorm等方法,进一步增强深层网络表现。代码已开源。

原文摘要 · Abstract (English)

Graph Convolutional Networks (GCNs) suffer from severe performance degradation in deep architectures due to over-smoothing. While existing studies primarily attribute the over-smoothing to repeated applications of graph Laplacian operators, our empirical analysis reveals a critical yet overlooked factor: trainable linear transformations in GCNs significantly exacerbate feature collapse, even at moderate depths (e.g., 8 layers). In contrast, Simplified Graph Convolution (SGC), which removes these transformations, maintains stable feature diversity up to 32 layers, highlighting linear transformations' dual role in facilitating expressive power and inducing over-smoothing. However, completely removing linear transformations weakens the model's expressive capacity. To address this trade-off, we propose Layer-wise Gradual Training (LGT), a novel training strategy that progressively builds deep GCNs while preserving their expressiveness. LGT integrates three complementary components: (1) layer-wise training to stabilize optimization from shallow to deep layers, (2) low-rank adaptation to fine-tune shallow layers and accelerate training, and (3) identity initialization to ensure smooth integration of new layers and accelerate convergence. Extensive experiments on benchmark datasets demonstrate that LGT achieves state-of-the-art performance on vanilla GCN, significantly improving accuracy even in 32-layer settings. Moreover, as a training method, LGT can be seamlessly combined with existing methods such as PairNorm and ContraNorm, further enhancing their performance in deeper networks. LGT offers a general, architecture-agnostic training framework for scalable deep GCNs. The code is available at [https://github.com/jfklasdfj/LGT_GCN].

图神经网络深度学习过平滑训练策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。