提出谱注意力机制，让图去噪更精准，尤其适合谱结构多变的图数据。
Graph Convolutional Attention: A Spectral Perspective on Graph Denoising and Diffusion
- 用输入图的谱信息设计新注意力，直接实现谱域去噪
- 在合成与真实数据上显著提升去噪效果，增益与谱多样性正相关
- 无需复杂结构特征或特征分解，推理更快且性能不降
图去噪是图学习的基础任务，也是图扩散模型的核心操作。基于注意力的架构如图变压器近期在图去噪中展现出潜力，但对其原理理解仍有限，尚不清楚标准注意力是否为最优机制。本文发现，在去噪目标下，线性注意力次优，仅能学习训练分布上的平均谱去噪滤波器，存在根本局限——因图在分布中常呈现多样谱结构。为此，我们提出谱注意力（Spectral Attention），直接利用输入图的谱信息，可证明其性能优于线性注意力，差距由分布的谱多样性决定。进一步推导出图卷积注意力（GCA），一种实际可用且置换等变的实现，通过图滤波查询与键实现谱去噪。对随机块模型，GCA可精确匹配理想谱注意力。此外，注意力后的softmax操作可通过近似将噪声特征向量投影到干净特征空间，提供额外去噪能力。实验表明，用GCA替代线性注意力，在合成与真实数据上持续提升图去噪与扩散性能，增益与谱多样性强相关。在DiGress中，GCA达到标准图变压器性能，无需计算昂贵结构特征；结合最近提出的PEARL位置编码，还避免显式特征分解，实现更快推理且质量不变。代码见：github.com/shervinkhalafi/graph_conv_att
原文摘要 · Abstract (English)
Denoising graphs is a fundamental problem in graph learning and the core operation of graph diffusion models. Attention-based architectures like graph transformers have recently shown promise in denoising graphs. However, our principled understanding of attention-based graph denoising remains limited, making it unclear whether standard attention is the right mechanism for this task. Here we show that, under a denoising objective, linear attention is suboptimal and can only learn an average spectral denoising filter over the training distribution. This creates a fundamental limitation as graphs often vary spectrally across the distribution. To overcome this limitation, we introduce Spectral Attention, which directly utilizes the input graph spectrum and provably outperforms linear attention by a margin governed by the spectral diversity of the distribution. We then derive Graph Convolutional Attention (GCA), a practical and permutation-equivariant realization of this idea that implements spectral denoising through graph-filtered queries and keys. For stochastic block models, GCA provably matches the idealized Spectral Attention mechanism. We further show that the softmax operation, that follows the attention, provides additional denoising by approximately projecting noisy eigenvectors onto the clean eigenspace. Empirically, replacing linear attention with GCA consistently improves graph denoising and diffusion on synthetic and real datasets, with gains strongly correlated with spectral diversity. In DiGress, GCA matches standard graph-transformer performance without computing expensive structural features, and when combined with the recently proposed PEARL positional encodings, avoids explicit eigendecomposition computations resulting in faster inference without degrading quality. The code can be found here: github.com/shervinkhalafi/graph_conv_att
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。