arXiv:2509.15024cs.LGcs.AI2025-09

提出AGCN模型,让Transformer在图聚类中既抓全局又保局部结构。

Attention Beyond Neighborhoods: Reviving Transformer for Graph Clustering

  • 将注意力机制直接嵌入图结构,兼顾全局与局部信息
  • 在7个基准数据集上超越现有方法,最高提升6.2个百分点
  • 适合对图聚类精度要求高的科研与工业场景

注意力机制在现代神经网络中扮演核心角色,但在图结构数据上的应用仍不充分,尤其在图聚类任务中表现不如图神经网络(GNN)。GNN过度依赖邻域聚合,导致节点表示同质化;而Transformer过度全局化,忽视有意义的局部模式。为此,本文通过实证分析揭示了GNN与Transformer在图聚类中的互补缺陷,提出新型注意力图聚类网络AGCN,将图结构重新定义为注意力对象。该模型直接将注意力机制嵌入图结构,实现高效全局信息提取并保留局部拓扑敏感性。理论分析对比了其与GNN和Transformer的行为差异,并引入两项创新:(1) 基于键值缓存的计算优化机制,(2) 成对边距对比损失,增强注意力空间的判别能力。大量实验表明,AGCN在7个基准数据集上均优于当前最优方法,性能最高提升6.2%。

原文摘要 · Abstract (English)

Attention mechanisms have become a cornerstone in modern neural networks, driving breakthroughs across diverse domains. However, their application to graph structured data, where capturing topological connections is essential, remains underexplored and underperforming compared to Graph Neural Networks (GNNs), particularly in the graph clustering task. GNN tends to overemphasize neighborhood aggregation, leading to a homogenization of node representations. Conversely, Transformer tends to over globalize, highlighting distant nodes at the expense of meaningful local patterns. This dichotomy raises a key question: Is attention inherently redundant for unsupervised graph learning? To address this, we conduct a comprehensive empirical analysis, uncovering the complementary weaknesses of GNN and Transformer in graph clustering. Motivated by these insights, we propose the Attentive Graph Clustering Network (AGCN) a novel architecture that reinterprets the notion that graph is attention. AGCN directly embeds the attention mechanism into the graph structure, enabling effective global information extraction while maintaining sensitivity to local topological cues. Our framework incorporates theoretical analysis to contrast AGCN behavior with GNN and Transformer and introduces two innovations: (1) a KV cache mechanism to improve computational efficiency, and (2) a pairwise margin contrastive loss to boost the discriminative capacity of the attention space. Extensive experimental results demonstrate that AGCN outperforms state-of-the-art methods.

图聚类注意力机制Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。