用九维积分特征系统分类激活函数，实现稳定性和表达力的可证明设计。
Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learning
- 构建九维积分签名，融合统计、斜率与光滑性指标，系统分类激活函数。
- 理论证明不同家族（饱和/线性/光滑）的稳定性差异，给出显式下降常数。
- 适合追求模型稳定性与可解释性的深度学习研究者参考使用。
激活函数决定神经网络的表达能力和稳定性，但现有比较多为启发式。本文提出一个严格框架，通过九维积分签名 S_sigma(phi) 对其进行分类，包含高斯传播统计量（m1, g1, g2, m2, eta）、渐近斜率（alpha_plus, alpha_minus）及正则性度量（TV(phi'), C(phi)）。该分类体系建立适定性，满足仿射参数化下的偏置不变律，并在有界斜率变化下保持封闭性。动力学分析导出带有显式下降常数的李雅普诺夫定理，通过 (m2', g2) 识别方差稳定性区域。从核视角推导出无维度的黑塞矩阵界，并将光滑性与 phi' 的有界变差相联系。应用该框架对八种标准激活函数（ReLU、leaky-ReLU、tanh、sigmoid、Swish、GELU、Mish、TeLU）进行分类，证明了饱和型、线性增长型与光滑型之间的清晰差异。数值上通过高斯-埃尔米特和蒙特卡洛验证理论预测。本框架为激活函数设计提供原则性指导，使选择从试错转向可证明的稳定性和核条件优化。
原文摘要 · Abstract (English)
Activation functions govern the expressivity and stability of neural networks, yet existing comparisons remain largely heuristic. We propose a rigorous framework for their classification via a nine-dimensional integral signature S_sigma(phi), combining Gaussian propagation statistics (m1, g1, g2, m2, eta), asymptotic slopes (alpha_plus, alpha_minus), and regularity measures (TV(phi'), C(phi)). This taxonomy establishes well-posedness, affine reparameterization laws with bias, and closure under bounded slope variation. Dynamical analysis yields Lyapunov theorems with explicit descent constants and identifies variance stability regions through (m2', g2). From a kernel perspective, we derive dimension-free Hessian bounds and connect smoothness to bounded variation of phi'. Applying the framework, we classify eight standard activations (ReLU, leaky-ReLU, tanh, sigmoid, Swish, GELU, Mish, TeLU), proving sharp distinctions between saturating, linear-growth, and smooth families. Numerical Gauss-Hermite and Monte Carlo validation confirms theoretical predictions. Our framework provides principled design guidance, moving activation choice from trial-and-error to provable stability and kernel conditioning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。