提出噪声掩码方法,让简化版图神经网络更深层、更高效且不丢失精度。
Towards Scalable and Deep Graph Neural Networks via Noise Masking
- 通过随机游走加噪声掩码,解决深层图网络过平滑问题。
- 在六个真实数据集上,性能优于原方法,兼具高准确率与低计算开销。
- 可直接插入现有简化模型,适合大规模图数据任务的开发者使用。
近年来,图神经网络(GNN)在众多图挖掘任务中取得显著成果。然而,由于训练过程中重复的特征传播和非线性变换带来高昂的计算与存储成本,将其扩展至大规模图仍具挑战。当前主流方法采用模型简化策略:仅在预处理阶段执行一次传播(P),再以不同方式组合感受野(C),最后输入简单模型以提升性能。尽管这些方法具备高预测能力与可扩展性,但仍存在两方面局限:其一,现有方法多从模型角度探索不同的组合方式,忽视了从数据视角出发的传播深度增加导致的性能下降问题,即过平滑现象;其二,预处理开销占整个流程的绝大部分时间,尤其对大规模图而言。为此,本文提出一种即插即用模块——随机游走噪声掩码(RMask),兼容现有模型简化工作,使深层GNN得以实现同时保持可扩展性。不同于以往方法,我们关注连续传播过程,发现每次传播内部存在的噪声是造成过平滑的根本原因,并设计高效的掩码机制予以消除。在六个真实世界数据集上的实验表明,加入RMask的模型简化方法相较于原始版本表现更优,能在准确率与效率间取得良好平衡。
原文摘要 · Abstract (English)
In recent years, Graph Neural Networks (GNNs) have achieved remarkable success in many graph mining tasks. However, scaling them to large graphs is challenging due to the high computational and storage costs of repeated feature propagation and non-linear transformation during training. One commonly employed approach to address this challenge is model-simplification, which only executes the Propagation (P) once in the pre-processing, and Combine (C) these receptive fields in different ways and then feed them into a simple model for better performance. Despite their high predictive performance and scalability, these methods still face two limitations. First, existing approaches mainly focus on exploring different C methods from the model perspective, neglecting the crucial problem of performance degradation with increasing P depth from the data-centric perspective, known as the over-smoothing problem. Second, pre-processing overhead takes up most of the end-to-end processing time, especially for large-scale graphs. To address these limitations, we present random walk with noise masking (RMask), a plug-and-play module compatible with the existing model-simplification works. This module enables the exploration of deeper GNNs while preserving their scalability. Unlike the previous model-simplification works, we focus on continuous P and found that the noise existing inside each P is the cause of the over-smoothing issue, and use the efficient masking mechanism to eliminate them. Experimental results on six real-world datasets demonstrate that model-simplification works equipped with RMask yield superior performance compared to their original version and can make a good trade-off between accuracy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。