提出gLSTM提升图神经网络存储能力，缓解信息挤压问题
gLSTM: Mitigating Over-Squashing by Increasing Storage Capacity
- 引入存储容量视角重看信息挤压，设计新任务验证瓶颈存在
- 基于序列建模思想改进图网络，显著提升节点信息承载能力
- 在合成任务与真实图数据上均表现优异，适合复杂图结构建模
图神经网络通过消息传递机制利用图结构传递节点间信息，但普遍存在过挤压问题：大量邻域信息被压缩成固定大小向量，形成信息瓶颈。本文从模型存储与检索容量角度重新审视该现象，定义为节点表示中可保存以备后续使用的最大信息量。我们指出现有评估方法的局限性，并提出一种新的合成任务，证明信息瓶颈会饱和存储容量。受序列建模中关联记忆、快速权重编程及xLSTM启发，我们设计了一种新型GNN架构，显著增强存储能力。实验表明，该架构在自定义容量任务和多个真实世界图基准测试中均表现优异。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) leverage the graph structure to transmit information between nodes, typically through the message-passing mechanism. While these models have found a wide variety of applications, they are known to suffer from over-squashing, where information from a large receptive field of node representations is collapsed into a single fixed sized vector, resulting in an information bottleneck. In this paper, we re-examine the over-squashing phenomenon through the lens of model storage and retrieval capacity, which we define as the amount of information that can be stored in a node's representation for later use. We study some of the limitations of existing tasks used to measure over-squashing and introduce a new synthetic task to demonstrate that an information bottleneck can saturate this capacity. Furthermore, we adapt ideas from the sequence modeling literature on associative memories, fast weight programmers, and the xLSTM model to develop a novel GNN architecture with improved capacity. We demonstrate strong performance of this architecture both on our capacity synthetic task, as well as a range of real-world graph benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。