RADE通过随机增删边同时解决GNN过拟合与长程信息压缩问题。
RADE: Random Add-Drop Edge as a Regularizer
- 随机增删边联合优化，兼顾正则化与长程通信。
- 实验显示显著降低过拟合并缓解信息压缩现象。
- 自适应调整增删率，实际使用无需调参。
图神经网络（GNN）存在过拟合和长程信息过压缩的问题。随机图增强（如删除边）虽能缓解过拟合，但引入训练-推理不一致，且无法改善过压缩；而重连方法虽提升连通性以缓解过压缩，却不具备正则化能力。本文提出随机增删边（RADE），一种联合删除与添加边的随机图增强方法，可同时应对过拟合与过压缩。RADE在理论上保证训练与推理对齐，使随机增强实现正则化而不产生分布偏移，同时支持推理时的长程通信。我们进一步提出并研究一种小批量梯度范数平衡算法，动态调节删除与添加率，使RADE在实践中实现超参数自由。在节点分类与图分类基准测试中，RADE表现优异，显著降低过拟合并缓解过压缩。消融实验证明了训练-推理对齐、自适应率选择及增删边互补作用的有效性。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) suffer from overfitting and over-squashing of long-range information. Stochastic graph augmentations (e.g., edge deletion) regularize training against overfitting but can introduce train-inference misalignment and do not improve over-squashing. In contrast, rewiring methods improve connectivity to mitigate over-squashing, but are not designed to regularize training. We propose Random Add-Drop Edge (RADE), a stochastic graph augmentation method that jointly drops and adds edges to address both overfitting and over-squashing simultaneously. RADE is provably designed to align training and inference so that random augmentations regularize training without distribution shift, while supporting long-range communication at inference. We further propose and study a mini-batch gradient-norm balancing algorithm that adapts deletion and addition rates during training, rendering RADE hyperparameter-free in practice. Experiments on node- and graph-classification benchmarks show that RADE is a strong regularizer and mitigates over-squashing. Ablations support the roles of train-inference alignment, adaptive rate selection, and the complementary effects of random edge deletion and edge addition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。