用信息论优化图神经网络的宽度与深度，缓解信息压缩问题。
How Wide and How Deep? Mitigating Over-Squashing of GNNs via Channel Capacity Constrained Estimation
- 基于信息论建模图网络为通信信道，定量设计隐藏维度与层数。
- 实验证明增加隐藏维度可缓解信息压缩，层数影响更复杂。
- 适合研究图神经网络表示学习与架构优化的读者。
现有图神经网络通常采用启发式方法设定隐藏维度和传播深度，导致信息在传播过程中严重丢失，即过压缩现象。为此，本文提出通道容量约束估计（C3E）框架，将隐藏维度与深度的选择建模为基于信息论的非线性规划问题。通过将谱图神经网络视为通信信道，该方法直接将通道容量与隐藏维度、传播深度、传播机制及图结构关联。在九个公开数据集上的大量实验表明，由C3E估算的隐藏维度与深度能有效缓解过压缩，持续提升表示学习性能。实验结果显示，过压缩源于表示矩阵中信息的累积压缩；同时发现，增加隐藏维度确实可缓解信息压缩，而传播深度的作用更为复杂，揭示了信息压缩与表示复杂度之间的根本平衡。
原文摘要 · Abstract (English)
Existing graph neural networks typically rely on heuristic choices for hidden dimensions and propagation depths, which often lead to severe information loss during propagation, known as over-squashing. To address this issue, we propose Channel Capacity Constrained Estimation (C3E), a novel framework that formulates the selection of hidden dimensions and depth as a nonlinear programming problem grounded in information theory. Through modeling spectral graph neural networks as communication channels, our approach directly connects channel capacity to hidden dimensions, propagation depth, propagation mechanism, and graph structure. Extensive experiments on nine public datasets demonstrate that hidden dimensions and depths estimated by C3E can mitigate over-squashing and consistently improve representation learning. Experimental results show that over-squashing occurs due to the cumulative compression of information in representation matrices. Furthermore, our findings show that increasing hidden dimensions indeed mitigate information compression, while the role of propagation depth is more nuanced, uncovering a fundamental balance between information compression and representation complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。