arXiv:2501.13456cs.LGcs.AI2025-01ICLR被引 5

用数学构造提升图神经网络注意力评分能力,性能最高提升20%以上。

KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks

  • 将柯尔莫哥洛夫-阿诺德网络引入注意力评分,增强表达能力。
  • 在有限参数下,新方法排名误差上限近乎无限,优于传统线性与MLP方法。
  • 可通用适配多数注意力图神经网络,适合追求性能提升的研究者。

近年来,带有注意力机制的图神经网络(GNN)已成为先进GNN模型的重要范式。然而,对邻居节点评分这一关键过程的理解仍不充分,导致许多现有注意力GNN表现不佳。本文统一了当前注意力GNN的评分函数,并提出柯尔莫哥洛夫-阿诺德注意力(KAA),将柯尔莫哥洛夫-阿诺德网络(KAN)架构融入评分过程。KAA显著提升了评分函数的整体性能,可应用于几乎所有现有注意力GNN。为定量评估不同评分函数的表达能力,我们引入最大排名距离(MRD),估算其在节点重要性排序中的误差上界。分析表明,在参数受限且宽度深度受限条件下,基于线性变换和MLP的评分函数具有有限表达能力;而所提出的KAA即使仅使用单层由零阶B样条参数化的KAN,也展现出近乎无限的表达能力。在多种骨干模型上进行的大量实验显示,经KAA增强的评分函数在节点级和图级任务中均持续优于原模型,某些情况下性能提升超过20%。

原文摘要 · Abstract (English)

Graph neural networks (GNNs) with attention mechanisms, often referred to as attentive GNNs, have emerged as a prominent paradigm in advanced GNN models in recent years. However, our understanding of the critical process of scoring neighbor nodes remains limited, leading to the underperformance of many existing attentive GNNs. In this paper, we unify the scoring functions of current attentive GNNs and propose Kolmogorov-Arnold Attention (KAA), which integrates the Kolmogorov-Arnold Network (KAN) architecture into the scoring process. KAA enhances the performance of scoring functions across the board and can be applied to nearly all existing attentive GNNs. To compare the expressive power of KAA with other scoring functions, we introduce Maximum Ranking Distance (MRD) to quantitatively estimate their upper bounds in ranking errors for node importance. Our analysis reveals that, under limited parameters and constraints on width and depth, both linear transformation-based and MLP-based scoring functions exhibit finite expressive power. In contrast, our proposed KAA, even with a single-layer KAN parameterized by zero-order B-spline functions, demonstrates nearly infinite expressive power. Extensive experiments on both node-level and graph-level tasks using various backbone models show that KAA-enhanced scoring functions consistently outperform their original counterparts, achieving performance improvements of over 20% in some cases.

图神经网络注意力机制KAN性能提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。