综述图Transformer架构、理论与应用,助你快速选型。
A Survey of Graph Transformers: Architectures, Theories and Applications
- 按结构处理策略分类图Transformer架构,涵盖分词、编码等方法。
- 对比不同架构表达能力,揭示其与图学习算法的内在联系。
- 提供实用指南表,匹配架构组件与四类图结构,便于实践选型。
图Transformer(GTs)通过克服图神经网络(GNNs)的过平滑和过挤压等固有局限,展现出强大的图结构建模能力。近期研究提出了多样化的架构设计,提升了可解释性,并拓展了实际应用。本文系统综述图Transformer,涵盖其架构、理论基础与应用。首先,根据结构信息处理策略对图Transformer架构进行分类,包括图分词、位置编码、结构感知注意力与模型集成。其次,从理论角度分析各类架构的表达能力,并与其它先进图学习算法对比,揭示其关联。在应用方面,文献围绕四类图组织形式组织:关系型、几何型、动态型与异构型。最后,通过一份实用指南表,按采用频率将架构组件映射至这些图形式,帮助实践者针对特定输入结构缩小设计选择范围。文章还讨论当前挑战与未来研究方向。
原文摘要 · Abstract (English)
Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing. Recent studies have proposed diverse architectures, enhanced explainability, and practical applications for Graph Transformers. In light of these rapid developments, we conduct a comprehensive review of Graph Transformers, covering aspects such as their architectures, theoretical foundations, and applications. In this survey, we first categorize the architecture of Graph Transformers according to their strategies for processing structural information, including graph tokenization, positional encoding, structure-aware attention, and model ensemble. Then, from the theoretical perspective, we examine the expressivity of Graph Transformers in various discussed architectures and contrast them with other advanced graph learning algorithms to discover their connections. For applications, we organize the literature around four graph organization forms, from relational, geometric, dynamic to heterogeneous. A Practical Guidance table then maps architectural components to these graph forms by adoption frequency, so practitioners can narrow down which design families to consider for a given input structure. Lastly, we will discuss the current challenges and prospective directions in Graph Transformers for potential future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。