发现图神经网络中节点信息被过度稀释，提出新方法缓解此问题。
Understanding and Tackling Over-Dilution in Graph Neural Networks
- 提出'过稀释'概念，量化节点属性与节点间信息衰减。
- 引入基于Transformer的NATR模型，显著提升单层信息保留能力。
- 适合研究图神经网络表示学习与架构优化的学者参考。
消息传递神经网络（MPNNs）在图机器学习中占据核心地位，但因其数据结构不规则，常出现过平滑和过挤压等非预期行为。本文深入探讨了此前被忽视的局限性，发现即使在单层内，单个节点的特定信息也可能显著稀释。为此，我们提出‘过稀释’概念，并定义了两个稀释因子：用于属性级的节点内稀释与用于节点级表示的节点间稀释。我们进一步设计了一种基于Transformer的解决方案NATR，有效缓解过稀释问题，并可与现有节点嵌入方法（如MPNN）互补。实验结果表明，该方法提升了图表示的丰富性。代码与补充材料已公开于https://github.com/LeeJunHyun/NATR。
原文摘要 · Abstract (English)
Message Passing Neural Networks (MPNNs) hold a key position in machine learning on graphs, but they struggle with unintended behaviors, such as over-smoothing and over-squashing, due to irregular data structures. The observation and formulation of these limitations have become foundational in constructing more informative graph representations. In this paper, we delve into the limitations of MPNNs, focusing on aspects that have previously been overlooked. Our observations reveal that even within a single layer, the information specific to an individual node can become significantly diluted. To delve into this phenomenon in depth, we present the concept of Over-dilution and formulate it with two dilution factors: intra-node dilution for attribute-level and inter-node dilution for node-level representations. We also introduce a transformer-based solution that alleviates over-dilution and complements existing node embedding methods like MPNNs. Our findings provide new insights and contribute to the development of informative representations. The implementation and supplementary materials are publicly available at https://github.com/LeeJunHyun/NATR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。