发现图神经网络短距离也会出现信息挤压，挑战了传统认知。
Short-Range Oversquashing
- 揭示短距离也存在信息瓶颈，与长距离无关
- 虚拟节点无法解决短距离瓶颈问题
- 图注意力模型在短距离任务中表现更优
消息传递神经网络（MPNNs）在图学习中广泛应用，但其处理长程信息的能力受限于信息挤压现象。部分研究主张用图变压器替代MPNN，另一些则建议通过虚拟节点或重连技术缓解该问题。本文发现，信息挤压不仅存在于长距离任务，也会出现在短距离问题中。这一发现使我们能够区分两种不同机制：（1）即使在低范围设置下也会出现的信息瓶颈现象；（2）与长距离任务紧密相关的梯度消失现象。我们进一步表明，现有解释无法涵盖短距离瓶颈效应，且添加虚拟节点无法缓解该问题。相比之下，图变压器在这些任务中表现良好，说明其相较专用MPNN更能有效应对信息挤压问题。
原文摘要 · Abstract (English)
Message Passing Neural Networks (MPNNs) are widely used for learning on graphs, but their ability to process long-range information is limited by the phenomenon of oversquashing. This limitation has led some researchers to advocate Graph Transformers as a better alternative, whereas others suggest that it can be mitigated within the MPNN framework, using virtual nodes or other rewiring techniques. In this work, we demonstrate that oversquashing is not limited to long-range tasks, but can also arise in short-range problems. This observation allows us to disentangle two distinct mechanisms underlying oversquashing: (1) the bottleneck phenomenon, which can arise even in low-range settings, and (2) the vanishing gradient phenomenon, which is closely associated with long-range tasks. We further show that the short-range bottleneck effect is not captured by existing explanations for oversquashing, and that adding virtual nodes does not resolve it. In contrast, transformers do succeed in such tasks, positioning them as the more compelling solution to oversquashing, compared to specialized MPNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。