揭示时空图神经网络中信息挤压的新问题
Over-squashing in Spatiotemporal Graph Neural Networks
- 提出时空图神经网络的信息挤压理论框架
- 发现卷积型模型更倾向远时序信息传播
- 为高效模型设计提供理论依据,适合相关研究者
图神经网络在多个领域取得显著成功,但近期理论研究表明其信息传播存在根本性局限,如过挤压现象——远距离节点难以有效交换信息。尽管静态图场景下已有广泛研究,但时空图神经网络(STGNNs)中的该问题尚未被探索。由于时间维度的引入,需传播的信息量显著增加,加剧了这一挑战。本文首次形式化了时空过挤压问题,揭示其与静态情形的不同特征。分析表明,反直觉地,卷积型STGNN更倾向于从时间上相距较远的点传播信息,而非邻近时间步。此外,我们证明采用时间-空间或时间后空间处理范式的架构均受此现象同等影响,为计算高效的实现提供了理论支持。在合成及真实数据集上的验证结果深化了对模型运行机制的理解,并为更有效的设计提供了原则性指导。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have achieved remarkable success across various domains. However, recent theoretical advances have identified fundamental limitations in their information propagation capabilities, such as over-squashing, where distant nodes fail to effectively exchange information. While extensively studied in static contexts, this issue remains unexplored in Spatiotemporal GNNs (STGNNs), which process sequences associated with graph nodes. Nonetheless, the temporal dimension amplifies this challenge by increasing the information that must be propagated. In this work, we formalize the spatiotemporal over-squashing problem and demonstrate its distinct characteristics compared to the static case. Our analysis reveals that, counterintuitively, convolutional STGNNs favor information propagation from points temporally distant rather than close in time. Moreover, we prove that architectures that follow either time-and-space or time-then-space processing paradigms are equally affected by this phenomenon, providing theoretical justification for computationally efficient implementations. We validate our findings on synthetic and real-world datasets, providing deeper insights into their operational dynamics and principled guidance for more effective designs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。