arXiv:2606.14022cs.LG2026-06

将度量尺度置于层归一化后,可显著提升图神经网络在关键任务上的表现。

Placing Degree Scales After LayerNorm

论文配图:Placing Degree Scales After LayerNorm
图 1 · 摘自论文原文
  • 在层归一化后添加反度量缩放,让节点度信息有效传递至输出层。
  • 在影响最大化、网络拆解等任务上,性能全面超越原模型且无需调参。
  • 方法简单高效,适合需要度敏感决策的图学习场景。

图神经网络广泛用于学习图上的节点选择策略,多数采用堆叠图注意力(GAT)模块并搭配层归一化(LayerNorm)。在依赖节点度的任务中,层归一化会消除模型所需的度信号,导致节点排序失准。现有工作多通过重构归一化或聚合机制来解决,但未思考度量尺度应置于层归一化前还是后。本文基于层归一化的代数特性揭示:若在层归一化前加入正的节点尺度,该尺度会被归一化过程抵消;而若在层归一化后加入,则可完整保留并影响最终输出。据此提出 PostDeg——一种无需参数的反度量缩放机制,仅需将逆度函数乘于层归一化后的表示。在不同规模评估中,PostDeg 均优于基线模型,在影响力最大化、网络拆解和最大独立集任务上表现突出。对比实验表明,位置必须在层归一化之后,且尺度必须与度相关,单一常数尺度在重尾图外无法生效。尺度具体形式影响较小,推荐使用无需调参的 PostDeg。

原文摘要 · Abstract (English)

Graph neural networks (GNNs) are widely used to learn node-selection policies on graphs, and most stack graph attention (GAT) blocks with LayerNorm. On degree-sensitive tasks, LayerNorm tends to remove the degree signal these models need to rank nodes. Much recent work addresses this by redesigning normalizers or aggregators, which changes what these components compute but does not ask where, relative to LayerNorm, a degree scale should be applied. In this paper, we show that the answer follows from a single algebraic fact about LayerNorm. When a positive per-node scale is applied before LayerNorm, LayerNorm divides it out, and it never reaches the model's output. Applied after LayerNorm, the same scale comes through and reaches the score head as magnitude. From this placement rule we derive PostDeg, a parameter-free inverse-degree scale that we add as the single change to a fixed GAT backbone. PostDeg multiplies each node's normalized representation by an inverse function of its degree, and we compare it against controls in the same position. At every evaluation size, PostDeg improves over the LayerNorm backbone on influence maximization, network dismantling, and maximum independent set, and these controls show where the improvement comes from. The same scale before LayerNorm stays at the backbone, as the absorption identity predicts, and a constant scale after LayerNorm stays there too on all but the most heavy-tailed graphs, so the effect needs both the position after LayerNorm and a degree-dependent scale. The exact form of the scale matters much less, so we recommend PostDeg, which needs no tuning.

图神经网络层归一化度敏感GAT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。