用意见分歧机制解决图神经网络过平滑问题,提升深层模型性能。
Resolving Oversmoothing with Opinion Dissensus
- 引入非线性意见动态作为归纳偏置,打破线性消息传递的共识倾向。
- BIMP在100步以上仍保持表征差异,超越现有主流架构与缓解方法。
- 适合处理同质与异质图数据,梯度稳定且泛化能力强。
尽管图神经网络(GNN)已在非欧几里得领域取得成功,但深层GNN常因过平滑现象导致性能下降,即节点表示随网络深度趋于相似。本文将此现象类比于意见动态中的完全共识,并证明多种GNN的消息传递机制等价于线性意见动态模型,后者无论图结构如何均收敛至共识。由此提出行为启发式消息传递(BIMP)GNN,采用更通用的非线性意见动态模型,可设计为对一般输入收敛至分歧状态。实验表明,BIMP在超过100个时间步后仍有效抵抗过平滑,且持续优于现有架构,即使在加入过平滑缓解技术后亦然。此外,BIMP具备良好梯度行为,适用于同质与异质数据集。
原文摘要 · Abstract (English)
While graph neural networks (GNNs) have allowed researchers to successfully apply neural networks to non-Euclidean domains, deep GNNs often exhibit lower predictive performance than their shallow counterparts. This phenomena has been attributed in part to oversmoothing, the tendency of node representations to become increasingly similar with network depth. In this paper we introduce an analogy between oversmoothing in GNNs and consensus (i.e., perfect agreement) in the opinion dynamics literature. We show that the message passing algorithms of several GNN models are equivalent to linear opinion dynamics models which have been shown to converge to consensus for all inputs regardless of the graph structure. This new perspective on oversmoothing motivates the use of nonlinear opinion dynamics as an inductive bias in GNN models. In our Behavior-Inspired Message Passing (BIMP) GNN, we leverage the nonlinear opinion dynamics model which is more general than the linear opinion dynamics model, and can be designed to converge to dissensus for general inputs. Through extensive experiments we show that BIMP resists oversmoothing beyond 100 time steps and consistently outperforms existing architectures even when those architectures are amended with oversmoothing mitigation techniques. We also show that BIMP has several desirable properties including well behaved gradients and adaptability to homophilic and heterophilic datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。