用跨批次聚合和上下文门控，让状态空间模型高效学习大规模图数据。
COMBA: Cross Batch Aggregation for Learning Large Graphs with Context Gating State Space Models
- 引入图上下文门控，动态控制邻居信息聚合。
- 跨批次聚合训练，实现对大规模图的有效建模。
- 理论保证更低误差，适合超大规模图学习任务。
状态空间模型（SSMs）因其较低计算成本，成为建模序列长程依赖的有力工具。将SSMs拓展至图结构数据，尤其在大规模图上面临挑战：图数据需转化为序列，而庞大的图规模使该过程极不高效。本文提出COMBA，通过两个关键创新解决此问题：图上下文门控与跨批次聚合。图上下文指每个节点不同邻域层级的信息，上下文门控使模型能据此优化邻居聚合策略。对于每种上下文，COMBA采样节点成批，训练图神经网络（GNN），并实现跨批次信息聚合，从而支持大规模图学习。理论分析表明，跨批次聚合可保证比无聚合训练更低的误差。在基准数据集上的实验显示，COMBA显著优于基线方法。代码与数据集将公开共享。
原文摘要 · Abstract (English)
State space models (SSMs) have recently emerged for modeling long-range dependency in sequence data, with much simplified computational costs than modern alternatives, such as transformers. Advancing SMMs to graph structured data, especially for large graphs, is a significant challenge because SSMs are sequence models and the shear graph volumes make it very expensive to convert graphs as sequences for effective learning. In this paper, we propose COMBA to tackle large graph learning using state space models, with two key innovations: graph context gating and cross batch aggregation. Graph context refers to different hops of neighborhood for each node, and graph context gating allows COMBA to use such context to learn best control of neighbor aggregation. For each graph context, COMBA samples nodes as batches, and train a graph neural network (GNN), with information being aggregated cross batches, allowing COMBA to scale to large graphs. Our theoretical study asserts that cross-batch aggregation guarantees lower error than training GNN without aggregation. Experiments on benchmark networks demonstrate significant performance gains compared to baseline approaches. Code and benchmark datasets will be released for public access.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。