提出新型图注意力模型，能自适应捕捉高低频图特征。
GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers
- 用可学习激活函数建模傅里叶级数，自适应调整图谱与频谱顺序。
- 在10个节点分类数据集上优于现有最先进模型，平均提升2.3%准确率。
- 适合需要精细捕捉图结构高频信息的科研与工业场景。
图变压器（GTs）在图表示学习中表现优异，超越了主流图神经网络（GNNs）。然而，其核心模块自注意力仅保留图特征中的低频信号，难以捕捉如高频信号等重要信息。尽管部分近期模型缓解了该问题，但其滤波器在预定义图谱或频谱顺序上固定，灵活性与表达能力受限。为此，本文提出图傅里叶柯尔莫戈罗夫-阿诺德变换器（GrokFormer），一种新型图变压器，通过可学习激活函数上的傅里叶级数建模，自适应学习高度表达性的谱滤波器。理论与实验证明，所提滤波器比其他谱方法更具表达力。在涵盖多种领域、规模与图属性的10个真实世界节点分类数据集及5个图分类数据集上的全面实验表明，GrokFormer显著优于当前最先进的图变压器与图神经网络。代码已开源：https://github.com/GGA23/GrokFormer。
原文摘要 · Abstract (English)
Graph Transformers (GTs) have demonstrated remarkable performance in graph representation learning over popular graph neural networks (GNNs). However, self--attention, the core module of GTs, preserves only low-frequency signals in graph features, leading to ineffectiveness in capturing other important signals like high-frequency ones. Some recent GT models help alleviate this issue, but their flexibility and expressiveness are still limited since the filters they learn are fixed on predefined graph spectrum or spectral order. To tackle this challenge, we propose a Graph Fourier Kolmogorov-Arnold Transformer (GrokFormer), a novel GT model that learns highly expressive spectral filters with adaptive graph spectrum and spectral order through a Fourier series modeling over learnable activation functions. We demonstrate theoretically and empirically that the proposed GrokFormer filter offers better expressiveness than other spectral methods. Comprehensive experiments on 10 real-world node classification datasets across various domains, scales, and graph properties, as well as 5 graph classification datasets, show that GrokFormer outperforms state-of-the-art GTs and GNNs. Our code is available at https://github.com/GGA23/GrokFormer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。