通过拆分计算图防止图神经网络表示秩坍缩
Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graph
- 用多关系图拆分消息传递路径，避免特征通道信号重复
- 在多个有向无环图上操作，可保证节点表示线性独立
- 适合研究图神经网络泛化能力的学者参考
图神经网络（MPNN）在拟合复杂图函数时受限于大多数图卷积在所有特征通道上放大相同信号，即表示秩坍缩，而过平滑是其特例。现有方法多通过残差连接、门控机制、归一化或正则化等扩展通用消息传递方案来缓解过平滑。本文反其道而行之，直接针对问题根源，修改消息传递机制，利用多关系图交换不同类型的消息。我们提出一个充分条件以确保节点表示线性独立；作为实例，证明在多个有向无环图上操作总能满足该条件，并通过定义节点的严格偏序关系生成这些图。全面实验验证了在多关系图上操作能显著提升节点表示的丰富性。
原文摘要 · Abstract (English)
The ability of message-passing neural networks (MPNNs) to fit complex functions over graphs is limited as most graph convolutions amplify the same signal across all feature channels, a phenomenon known as rank collapse, and over-smoothing as a special case. Most approaches to mitigate over-smoothing extend common message-passing schemes, e.g., the graph convolutional network, by utilizing residual connections, gating mechanisms, normalization, or regularization techniques. Our work contrarily proposes to directly tackle the cause of this issue by modifying the message-passing scheme and exchanging different types of messages using multi-relational graphs. We identify a sufficient condition to ensure linearly independent node representations. As one instantion, we show that operating on multiple directed acyclic graphs always satisfies our condition and propose to obtain these by defining a strict partial ordering of the nodes. We conduct comprehensive experiments that confirm the benefits of operating on multi-relational graphs to achieve more informative node representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。