arXiv:2412.04738cs.LG2024-12被引 2

提出新型图注意力网络,让大图处理更快更高效。

DHIL-GT: Scalable Graph Transformer with Decoupled Hierarchy Labeling

  • 分离图计算阶段,用标签层级解耦全局计算
  • 预计算后训练阶段复杂度线性于节点和边数
  • 适合处理异质图,支持大规模批处理与高效显存利用

图注意力网络(Graph Transformer, GT)虽在图结构数据学习中表现优异,但其全局注意力机制的二次复杂度限制了其在大规模图上的应用。现有方法通过修改模型结构或编码层次信息提升可扩展性,但仍未解决图规模操作带来的计算瓶颈。本文提出DHIL-GT,通过预先完全解耦图计算至独立阶段,实现高效学习。该方法利用图标签层次结构提取层次信息,其蕴含的全局连接与局部性特征优于原始邻接关系,尤其适用于异质图等复杂模式。进一步设计子图采样与位置编码方案,在图标签基础上端到端预计算模型输入,使训练阶段无需进行图相关计算,显著提升小批量处理能力和GPU利用率。实验表明,DHIL-GT在大规模基准上兼具计算效率与顶尖性能,其预计算与训练阶段复杂度分别线性于图边数与节点数,显著优于现有可扩展图注意力模型。

原文摘要 · Abstract (English)

Graph Transformer (GT) has recently emerged as a promising neural network architecture for learning graph-structured data. However, its global attention mechanism with quadratic complexity concerning the graph scale prevents wider application to large graphs. While current methods attempt to enhance GT scalability by altering model architecture or encoding hierarchical graph data, our analysis reveals that these models still suffer from the computational bottleneck related to graph-scale operations. In this work, we target the GT scalability issue and propose DHIL-GT, a scalable Graph Transformer that simplifies network learning by fully decoupling the graph computation to a separate stage in advance. DHIL-GT effectively retrieves hierarchical information by exploiting the graph labeling technique, as we show that the graph label hierarchy is more informative than plain adjacency by offering global connections while promoting locality, and is particularly suitable for handling complex graph patterns such as heterophily. We further design subgraph sampling and positional encoding schemes for precomputing model input on top of graph labels in an end-to-end manner. The training stage thus favorably removes graph-related computations, leading to ideal mini-batch capability and GPU utilization. Notably, the precomputation and training processes of DHIL-GT achieve complexities linear to the number of graph edges and nodes, respectively. Extensive experiments demonstrate that DHIL-GT is efficient in terms of computational boost and mini-batch capability over existing scalable Graph Transformer designs on large-scale benchmarks, while achieving top-tier effectiveness on both homophilous and heterophilous graphs.

图神经网络注意力机制可扩展性异质图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。