arXiv:2412.08193cs.LG2024-12中稿 · ACM Web Conference…被引 6

GNNMoE融合专家模型与消息传递,提升节点分类的通用性与适应性。

Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification

  • 用专家机制动态选择适合的节点处理路径
  • 在多种图数据上实现低过平滑、抗噪声且高效计算
  • 适合处理异质图与大规模图的自适应分类任务

图神经网络在图表示学习中表现优异,但在异质数据和长距离依赖场景下表现不佳。图变压器虽通过自注意力缓解这些问题,却面临大规模图上的可扩展性与噪声敏感挑战。为此,我们提出GNNMoE,一种面向节点分类的通用模型架构。该架构将细粒度的消息传递操作与混合专家机制相结合,构建特征编码模块;通过引入软/硬门控层,为每个节点分配最合适的专家网络,增强模型表达能力与对不同图类型的适应性。此外,GNNMoE还集成自适应残差连接与增强型前馈网络模块,进一步提升节点表征能力。大量实验表明,GNNMoE在各类图数据上表现优异,有效缓解过平滑与全局噪声问题,显著提升模型鲁棒性与适应性,同时在大规模图上保持高效计算性能。

原文摘要 · Abstract (English)

Graph neural networks excel at graph representation learning but struggle with heterophilous data and long-range dependencies. And graph transformers address these issues through self-attention, yet face scalability and noise challenges on large-scale graphs. To overcome these limitations, we propose GNNMoE, a universal model architecture for node classification. This architecture flexibly combines fine-grained message-passing operations with a mixture-of-experts mechanism to build feature encoding blocks. Furthermore, by incorporating soft and hard gating layers to assign the most suitable expert networks to each node, we enhance the model's expressive power and adaptability to different graph types. In addition, we introduce adaptive residual connections and an enhanced FFN module into GNNMoE, further improving the expressiveness of node representation. Extensive experimental results demonstrate that GNNMoE performs exceptionally well across various types of graph data, effectively alleviating the over-smoothing issue and global noise, enhancing model robustness and adaptability, while also ensuring computational efficiency on large-scale graphs.

图神经网络专家模型节点分类自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。