提出Top-m注意力机制的轻量图神经网络,高效捕捉远距离节点关系。
TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning
- 融合顶m注意力与邻域聚合,兼顾局部与全局信息
- 在5个任务上超越现有方法,尤其在大规模图上效率更高
- 适用于需要高效建模长程依赖的复杂图分析场景
在深度学习领域,图神经网络(GNN)和图Transformer模型凭借优异性能和灵活架构,成为处理图数据的主流技术。传统GNN难以有效捕获远距离节点信息,而图Transformer虽擅长建模长距离关系,但在大规模图上存在计算与存储效率问题。为此,我们提出一种新型图神经网络架构,集成顶m注意力机制聚合组件与邻域聚合组件,有效提升每层对局部及扩展邻域相关信息的聚合能力。该方法不仅提高计算效率,还丰富节点特征,利于深入分析复杂图结构。为验证模型有效性,我们首次将该模型应用于引文情感预测这一新任务,构建专用引文网络ArXivNet,标注了引用的情感极性(正向、中性、负向)以支持深层情感分析。实验结果表明,该方法在节点分类、链接预测、情感预测、图回归和可视化等多任务中表现卓越,优于现有方法。
原文摘要 · Abstract (English)
In the field of deep learning, Graph Neural Networks (GNNs) and Graph Transformer models, with their outstanding performance and flexible architectural designs, have become leading technologies for processing structured data, especially graph data. Traditional GNNs often face challenges in capturing information from distant vertices effectively. In contrast, Graph Transformer models are particularly adept at managing long-distance node relationships. Despite these advantages, Graph Transformer models still encounter issues with computational and storage efficiency when scaled to large graph datasets. To address these challenges, we propose an innovative Graph Neural Network (GNN) architecture that integrates a Top-m attention mechanism aggregation component and a neighborhood aggregation component, effectively enhancing the model's ability to aggregate relevant information from both local and extended neighborhoods at each layer. This method not only improves computational efficiency but also enriches the node features, facilitating a deeper analysis of complex graph structures. Additionally, to assess the effectiveness of our proposed model, we have applied it to citation sentiment prediction, a novel task previously unexplored in the GNN field. Accordingly, we constructed a dedicated citation network, ArXivNet. In this dataset, we specifically annotated the sentiment polarity of the citations (positive, neutral, negative) to enable in-depth sentiment analysis. Our approach has shown superior performance across a variety of tasks including vertex classification, link prediction, sentiment prediction, graph regression, and visualization. It outperforms existing methods in terms of effectiveness, as demonstrated by experimental results on multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。