用正交反射器缓解图神经网络深度增加导致的节点表示过度平滑问题。
Exploring Oversmoothing with Householder Matrices
- 通过霍舒勒反射器更新节点嵌入，保持每层欧氏范数不变。
- 理论证明各层保留节点级范数，且对消息缩放和符号变化不变。
- 可动态调整节点间距离，适合深层图模型应用。
深层图神经网络（GNN）面临过度平滑问题：随着网络深度增加，节点表示因重复应用归一化图传播算子而逐渐坍缩至低信息子空间。本文提出霍舒勒图神经网络（HouseGNN）。不同于标准GCN直接更新隐藏状态，HouseGNN仅用邻域聚合消息估计反射方向，再通过霍舒勒反射器与GroupSort更新节点嵌入，形成分段正交层，确保每个节点、每层均保持欧氏范数。理论证明三个核心性质：(i) 每个内部层保持节点级欧氏范数；(ii) 霍舒勒反射器对消息缩放和符号变化不变；(iii) 节点间距离可通过节点级正交算子不匹配而改变。
原文摘要 · Abstract (English)
Deep graph neural networks(GNNs) suffer from oversmoothing- a progressive collapse of node representation towards a low information subspace as network depth increases because the normalized graph propagation operator is repeatedly applied directly to the hidden representations. In this work we study Householder Graph Neural Network (HouseGNN). Rather than updating the hidden state like standard GCN, HouseGNN uses the aggregated neighbourhood message solely to estimate a reflection direction; the node embedding is then updated by a Householder reflector followed by GroupSort, yielding a piecewise orthogonal layer that preserves Euclidean norm at every node and at every depth. We prove three core properties: (i) every internal layer preserves the node-wise Euclidean norm; (ii) the Householder reflector is scale scale and sign-invariant in the message; and (iii) pairwise distance between nodes can change through mismatch between node-wise orthogonal operators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。