用双路Mamba模型解决图神经网络深层过平滑问题
Dual Mamba for Node-Specific Representation Learning: Tackling Over-Smoothing with Selective State Space Modeling
- 引入局部与全局双Mamba模块,动态建模节点表征演化
- 在多个基准上显著提升深层图网络的节点区分度
- 适合研究深层图神经网络优化与状态空间模型应用
深度图神经网络中的过平滑问题仍是一大挑战,反复的消息传递导致节点表征趋于相似。现有方法如残差连接虽有缓解作用,但未能显式建模节点表征在各层间的节点特异性、渐进式演变过程,且忽略全局信息。为此,本文提出双路Mamba增强的图卷积网络(DMbaGCN),融合Mamba模型从局部和全局视角解决过平滑问题。该框架包含两个模块:局部状态演化Mamba(LSEMba)用于邻域聚合,利用Mamba的选择性状态空间建模捕捉节点表征的层间动态;全局上下文感知Mamba(GCAMba)则借助Mamba的全局注意力机制为每个节点引入全局上下文。通过结合两者,DMbaGCN有效增强了深层GNN中节点的可区分性,缓解过平滑。在多个基准数据集上的实验验证了方法的有效性与高效性。
原文摘要 · Abstract (English)
Over-smoothing remains a fundamental challenge in deep Graph Neural Networks (GNNs), where repeated message passing causes node representations to become indistinguishable. While existing solutions, such as residual connections and skip layers, alleviate this issue to some extent, they fail to explicitly model how node representations evolve in a node-specific and progressive manner across layers. Moreover, these methods do not take global information into account, which is also crucial for mitigating the over-smoothing problem. To address the aforementioned issues, in this work, we propose a Dual Mamba-enhanced Graph Convolutional Network (DMbaGCN), which is a novel framework that integrates Mamba into GNNs to address over-smoothing from both local and global perspectives. DMbaGCN consists of two modules: the Local State-Evolution Mamba (LSEMba) for local neighborhood aggregation and utilizing Mamba's selective state space modeling to capture node-specific representation dynamics across layers, and the Global Context-Aware Mamba (GCAMba) that leverages Mamba's global attention capabilities to incorporate global context for each node. By combining these components, DMbaGCN enhances node discriminability in deep GNNs, thereby mitigating over-smoothing. Extensive experiments on multiple benchmarks demonstrate the effectiveness and efficiency of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。