针对图数据规模差异导致的深度敏感问题,提出自适应专家混合模型。
DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts
- 采用多个带独立参数的专家网络,按图规模动态选择合适深度
- 在TU和OGB数据集上超越现有方法,提升图、节点、链接级任务性能
- 适合处理规模不一的真实图数据,尤其适用于多尺度图分析场景
图神经网络(GNN)在处理图结构数据方面日益流行。然而,同一数据集中图数据的规模差异显著,导致深度敏感性:小图只需较少层消息传递,大图则需更深网络以捕捉长程依赖和全局特征。现有方法普遍使用固定层数的GNN生成所有图的表示,忽视了这一问题。为此,我们提出深度自适应专家混合(DA-MoE)方法,对GNN主干进行两项改进:1)引入多个独立参数的专家网络,每个专家对应不同深度,实现对不同规模图信息的灵活聚合,有效缓解深度敏感性;2)用GNN代替线性投影构建门控网络,使门控机制能捕捉数据内部复杂模式与依赖关系。各专家专门学习不同尺度下的图模式。在TU数据集和开放图基准(OGB)上的全面实验表明,DA-MoE在图、节点和链接级分析任务中持续优于现有基线。代码已公开于https://github.com/Celin-Yao/DA-MoE。
原文摘要 · Abstract (English)
Graph neural networks (GNNs) are gaining popularity for processing graph-structured data. In real-world scenarios, graph data within the same dataset can vary significantly in scale. This variability leads to depth-sensitivity, where the optimal depth of GNN layers depends on the scale of the graph data. Empirically, fewer layers are sufficient for message passing in smaller graphs, while larger graphs typically require deeper networks to capture long-range dependencies and global features. However, existing methods generally use a fixed number of GNN layers to generate representations for all graphs, overlooking the depth-sensitivity issue in graph structure data. To address this challenge, we propose the depth adaptive mixture of expert (DA-MoE) method, which incorporates two main improvements to GNN backbone: \textbf{1)} DA-MoE employs different GNN layers, each considered an expert with its own parameters. Such a design allows the model to flexibly aggregate information at different scales, effectively addressing the depth-sensitivity issue in graph data. \textbf{2)} DA-MoE utilizes GNN to capture the structural information instead of the linear projections in the gating network. Thus, the gating network enables the model to capture complex patterns and dependencies within the data. By leveraging these improvements, each expert in DA-MoE specifically learns distinct graph patterns at different scales. Furthermore, comprehensive experiments on the TU dataset and open graph benchmark (OGB) have shown that DA-MoE consistently surpasses existing baselines on various tasks, including graph, node, and link-level analyses. The code are available at \url{https://github.com/Celin-Yao/DA-MoE}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。