提出酉卷积解决图神经网络过平滑问题,让深层模型更稳定。
Unitary convolutions for learning on graphs and groups
- 用酉矩阵设计卷积层,保持信息不变性防止过平滑
- 理论证明可避免图网络在几轮消息传递后失效
- 适合需要深层结构的图学习任务,如复杂关系建模
具有几何结构的数据在机器学习中普遍存在,常源于领域中的基本对称性,如图的置换不变性和图像的平移不变性。群卷积架构通过引入对称性作为归纳偏置,在应用中表现优异,但随着深度增加易出现不稳定性,且难以学习长程依赖。例如,图神经网络因节点表示趋同(过平滑)而失稳,可能在仅数轮消息传递后即失效,影响下游任务性能。本文提出并研究酉群卷积,支持更深网络并提升训练稳定性。重点研究图神经网络,证明酉图卷积可理论避免过平滑。实验结果表明,酉图卷积网络在基准数据集上性能媲美当前最优图神经网络。此外,我们还拓展分析一般酉卷积在群卷积架构中的稳定性作用。
原文摘要 · Abstract (English)
Data with geometric structure is ubiquitous in machine learning often arising from fundamental symmetries in a domain, such as permutation-invariance in graphs and translation-invariance in images. Group-convolutional architectures, which encode symmetries as inductive bias, have shown great success in applications, but can suffer from instabilities as their depth increases and often struggle to learn long range dependencies in data. For instance, graph neural networks experience instability due to the convergence of node representations (over-smoothing), which can occur after only a few iterations of message-passing, reducing their effectiveness in downstream tasks. Here, we propose and study unitary group convolutions, which allow for deeper networks that are more stable during training. The main focus of the paper are graph neural networks, where we show that unitary graph convolutions provably avoid over-smoothing. Our experimental results confirm that unitary graph convolutional networks achieve competitive performance on benchmark datasets compared to state-of-the-art graph neural networks. We complement our analysis of the graph domain with the study of general unitary convolutions and analyze their role in enhancing stability in general group convolutional architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。