arXiv:2602.01553cs.LGcs.AI2026-02

简单Transformer通过采样子图注意力,高效实现强链接预测。

Plain Transformers are Surprisingly Powerful Link Predictors

  • 用采样局部子图的自注意力替代手工设计结构特征
  • 在多个数据集上超越启发式GNN,参数量更少
  • 适合追求高效、可扩展链接预测的工程与研究者

链接预测是图机器学习的核心挑战,需捕捉复杂的拓扑依赖。尽管图神经网络(GNN)是主流方案,但先进方法常依赖显式结构启发式或内存密集型节点嵌入,难以泛化或扩展至大规模图。新兴图变压器(GTs)虽具潜力,却因复杂结构编码带来显著开销。本文提出PENCIL——一种仅含编码器的纯Transformer模型,以采样局部子图上的注意力取代人工先验,保持标准Transformer的可扩展性与硬件效率。实验与理论分析表明,PENCIL提取的结构信号比GNN更丰富,隐式推广了广泛启发式与子图表达能力。实证结果表明,其性能超越启发式增强的GNN,远优于基于ID嵌入的方案,且无需节点特征仍具竞争力。研究挑战了对复杂工程的依赖,证明简洁设计即可达成同等能力。代码已公开于https://github.com/quang-truong/pencil。

原文摘要 · Abstract (English)

Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies. While Graph Neural Networks (GNNs) are the standard solution, state-of-the-art pipelines often rely on explicit structural heuristics or memory-intensive node embeddings -- approaches that struggle to generalize or scale to massive graphs. Emerging Graph Transformers (GTs) offer a potential alternative but often incur significant overhead due to complex structural encodings, hindering their applications to large-scale link prediction. We challenge these sophisticated paradigms with PENCIL, an encoder-only plain Transformer that replaces hand-crafted priors with attention over sampled local subgraphs, retaining the scalability and hardware efficiency of standard Transformers. Through experimental and theoretical analysis, we show that PENCIL extracts richer structural signals than GNNs, implicitly generalizing a broad class of heuristics and subgraph-based expressivity. Empirically, PENCIL outperforms heuristic-informed GNNs and is far more parameter-efficient than ID-embedding--based alternatives, while remaining competitive across diverse benchmarks -- even without node features. Our results challenge the prevailing reliance on complex engineering techniques, demonstrating that simple design choices are potentially sufficient to achieve the same capabilities. Our code is publicly available at https://github.com/quang-truong/pencil.

图神经网络链接预测Transformer可扩展性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。