将图数据转化为序列，让Transformer直接处理图任务
Graph Tokenization for Bridging Graphs and Transformers
- 用可逆图序列化+BPE分词，生成保留图结构的离散序列
- 在14个图数据集上超越GNN和专用图Transformer，达最新水平
- 适合想用现成Transformer处理图数据的研究者
大型预训练Transformer的成功与其分词器密切相关，分词器将原始输入转换为离散符号。将此类模型扩展到图结构数据仍面临重大挑战。本文提出一种图分词框架，通过结合可逆图序列化（保持图信息）与字节对编码（BPE，大语言模型中广泛使用的分词器），生成图的序列表示。为更好捕捉结构信息，图序列化过程受图子结构全局统计引导，使频繁出现的子结构在序列中更常出现，从而被BPE合并为有意义的词汇单元。实验表明，该分词器使BERT等Transformer可直接应用于图基准任务而无需架构修改。所提方法在14个基准数据集上达到最先进性能，通常优于图神经网络及专用图Transformer。本工作弥合了图结构数据与序列模型生态之间的鸿沟。代码已公开于：https://github.com/BUPT-GAMMA/Graph-Tokenization-for-Bridging-Graphs-and-Transformers。
原文摘要 · Abstract (English)
The success of large pretrained Transformers is closely tied to tokenizers, which convert raw input into discrete symbols. Extending these models to graph-structured data remains a significant challenge. In this work, we introduce a graph tokenization framework that generates sequential representations of graphs by combining reversible graph serialization, which preserves graph information, with Byte Pair Encoding (BPE), a widely adopted tokenizer in large language models (LLMs). To better capture structural information, the graph serialization process is guided by global statistics of graph substructures, ensuring that frequently occurring substructures appear more often in the sequence and can be merged by BPE into meaningful tokens. Empirical results demonstrate that the proposed tokenizer enables Transformers such as BERT to be directly applied to graph benchmarks without architectural modifications. The proposed approach achieves state-of-the-art results on 14 benchmark datasets and frequently outperforms both graph neural networks and specialized graph transformers. This work bridges the gap between graph-structured data and the ecosystem of sequence models. Our code is available at \href{https://github.com/BUPT-GAMMA/Graph-Tokenization-for-Bridging-Graphs-and-Transformers}{\color{blue}here}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。