用Mamba思想解决图神经网络过平滑问题
Mamba-Based Graph Convolutional Networks: Tackling Over-smoothing with Selective State Space
- 引入选择性状态空间机制,动态调整邻域信息聚合
- 在多个基准数据集上显著缓解深层网络过平滑现象
- 为深度图神经网络设计提供新范式,适合研究架构创新者
图神经网络在各类图学习任务中表现优异,但随着模型加深,常出现过平滑问题,导致节点表示趋于一致而失去区分性。这源于传统GNN难以区分不同邻域信息的重要性。本文提出MbaGCN,受Mamba序列建模思想启发,构建新型图卷积架构,包含消息聚合层、选择性状态空间转移层和节点状态预测层。三者协同实现自适应邻域信息聚合,提升深层模型的灵活性与可扩展性。尽管在部分数据集上未全面超越现有方法,但其展示了将Mamba范式有效融入图表示学习的潜力。大量实验表明,MbaGCN为图神经网络研究开辟了新路径。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have shown great success in various graph-based learning tasks. However, it often faces the issue of over-smoothing as the model depth increases, which causes all node representations to converge to a single value and become indistinguishable. This issue stems from the inherent limitations of GNNs, which struggle to distinguish the importance of information from different neighborhoods. In this paper, we introduce MbaGCN, a novel graph convolutional architecture that draws inspiration from the Mamba paradigm-originally designed for sequence modeling. MbaGCN presents a new backbone for GNNs, consisting of three key components: the Message Aggregation Layer, the Selective State Space Transition Layer, and the Node State Prediction Layer. These components work in tandem to adaptively aggregate neighborhood information, providing greater flexibility and scalability for deep GNN models. While MbaGCN may not consistently outperform all existing methods on each dataset, it provides a foundational framework that demonstrates the effective integration of the Mamba paradigm into graph representation learning. Through extensive experiments on benchmark datasets, we demonstrate that MbaGCN paves the way for future advancements in graph neural network research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。