用分岔理论破解图神经网络过平滑难题，让深层模型保持信息丰富性。
Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
- 从分岔理论出发，揭示过平滑是稳定均质态的必然结果。
- 替换激活函数可诱发分岔，生成抗过平滑的非均质模式，幅度符合精确标度律。
- 提出可直接应用的初始化方法，显著提升真实数据集上的深层模型性能。
图神经网络通过迭代的消息传递学习节点表示。尽管强大，深层图神经网络会遭遇过平滑问题，即节点特征收敛至同质、无信息的状态。本文从分岔理论视角重新审视这一表征崩溃问题，将过平滑描述为收敛至稳定的‘均质不动点’。核心贡献是理论发现：用一类特殊函数替代标准单调激活（如ReLU），可打破该不良稳定性。借助Lyapunov-Schmidt约化，我们严格证明此替换会引发分岔，破坏均质状态并生成一对新的稳定非均质模式，能有效抵抗过平滑。理论预测了这些涌现模式的振幅存在精确而非平凡的标度律，实验中定量验证。最后，基于该理论推导出闭式分岔感知初始化，并在真实基准测试中展示了其有效性。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) learn node representations through iterative network-based message-passing. While powerful, deep GNNs suffer from oversmoothing, where node features converge to a homogeneous, non-informative state. We re-frame this problem of representational collapse from a \emph{bifurcation theory} perspective, characterizing oversmoothing as convergence to a stable ``homogeneous fixed point.'' Our central contribution is the theoretical discovery that this undesired stability can be broken by replacing standard monotone activations (e.g., ReLU) with a class of functions. Using Lyapunov-Schmidt reduction, we analytically prove that this substitution induces a bifurcation that destabilizes the homogeneous state and creates a new pair of stable, non-homogeneous \emph{patterns} that provably resist oversmoothing. Our theory predicts a precise, nontrivial scaling law for the amplitude of these emergent patterns, which we quantitatively validate in experiments. Finally, we demonstrate the practical utility of our theory by deriving a closed-form, bifurcation-aware initialization and showing its utility in real benchmark experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。