图Transformer通过位置编码实现小图训练大图泛化，理论支持高效扩展。
Size Transferability of Graph Transformers with Convolutional Positional Encodings
- 用卷积式位置编码将图Transformer与流形神经网络关联
- 小图训练的模型可证明泛化到大图，条件温和
- 适合大规模图数据的高效训练场景
Transformer在多个领域取得成功，推动了基于注意力机制的图变压器（Graph Transformers, GTs）发展。GTs的关键设计之一是使用基于图神经网络（GNN）的位置编码来融入结构信息。本文从图序列的流形极限模型视角研究GTs，建立了使用GNN位置编码的GTs与流形神经网络（MNNs）之间的理论联系。基于流形收敛下的GNN可迁移性结果，我们证明GTs继承了其位置编码的可迁移性。具体而言，在温和假设下，小图上训练的GTs能保证泛化到更大图。我们通过标准图基准上的大量实验验证理论，表明GTs具有与GNN相当的可扩展行为。为进一步展示实际效率，我们在地形最短路径距离估计任务中实现GTs，凸显可迁移GTs的优势。结果为理解GTs提供了新视角，并指明了大规模设置下高效训练的实际方向。
原文摘要 · Abstract (English)
Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for graph-structured data. A key design choice in GTs is the use of Graph Neural Network (GNN)-based positional encodings to incorporate structural information. In this work, we study GTs through the lens of manifold limit models for graph sequences and establish a theoretical connection between GTs with GNN positional encodings and Manifold Neural Networks (MNNs). Building on transferability results for GNNs under manifold convergence, we show that GTs inherit transferability guarantees from their positional encodings. In particular, GTs trained on small graphs provably generalize to larger graphs under mild assumptions. We complement our theory with extensive experiments on standard graph benchmarks, demonstrating that GTs exhibit scalable behavior on par with GNNs. To further show the efficiency in a real-world scenario, we implement GTs for shortest path distance estimation over terrains to better illustrate the efficiency of the transferable GTs. Our results provide new insights into the understanding of GTs and suggest practical directions for efficient training of GTs in large-scale settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。