将图傅里叶变换引入注意力机制,同时捕捉结构与频率信息提升图模型性能。
Graph Fourier Transformer with Structure-Frequency Information
- 通过图拉普拉斯特征值构建频域掩码,融合节点位置与邻域结构关系。
- 利用逆傅里叶变换提取高低频特征,构建能量矩阵优化注意力分布。
- 在多类图分类任务中超越主流GNN与图变压器,适合结构敏感任务研究者。
图变压器(GTs)在众多图结构任务中表现优异,但其自注意力机制忽略了图的泛化偏差。现有方法主要从位置编码、注意力偏置和相对距离等角度补偿该偏差,但仍存在性能不足的问题,且仅从结构视角出发。为此,本文提出Grafourierformer,创新性地将包含频域-结构信息的归纳偏置融入图变压器:通过图傅里叶变换对注意力矩阵进行处理,利用图拉普拉斯矩阵的特征值构建特征值掩码(反映节点位置与邻近节点的结构关系,以关注局部图细节),并应用逆傅里叶变换提取节点的高频与低频特征,计算高低频能量,构造节点频域能量矩阵,用于过滤特征值掩码。这使得注意力头能自适应区分全局趋势与局部细节,有效抑制冗余信息干扰。大量实验表明,Grafourierformer在多个基准数据集上持续优于基于GNN和GT的模型,在图分类与节点分类任务中表现更优。消融实验进一步验证了该方法的有效性与必要性。代码已公开于 https://github.com/Arichibald/Grafourierformer.git。
原文摘要 · Abstract (English)
Graph Transformers (GTs) have shown advantages in numerous graph structure tasks but their self-attention mechanism ignores the generalization bias of graphs, with existing methods mainly compensating for this bias from aspects like position encoding, attention bias and relative distance yet still having sub-optimal performance and being insufficient by only considering the structural perspective of generalization bias. To address this, this paper proposes Grafourierformer, which innovatively combines GT with inductive bias containing Frequency-Structure information by applying Graph Fourier Transform to the Attention Matrix: specifically, eigenvalues from the Graph Laplacian matrix are used to construct an Eigenvalue matrix mask (reflecting node positions and structural relationships with neighboring nodes to enable consideration of node range structural characteristics and focus on local graph details), and inverse Fourier transform is employed to extract node high-frequency and low-frequency features, calculate low-frequency and high-frequency energy, and construct a node frequency-energy matrix to filter the eigenvalue matrix mask, allowing attention heads to incorporate both graph structural information and node frequency information optimization, adaptively distinguish global trends from local details, and effectively suppress redundant information interference. Extensive experiments on various benchmarks show Grafourierformer consistently outperforms GNN and GT-based models in graph classification and node classification tasks, with ablation experiments further validating the effectiveness and necessity of the method. Codes are available at https://github.com/Arichibald/Grafourierformer.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。