证明了GELU网络能同时逼近函数及其任意阶导数
Approximation Capabilities of Feedforward Neural Networks with GELU Activations
- 基于构造性乘法逼近，实现多阶导数统一误差控制
- 对多项式、指数、倒数等函数均给出误差上界
- 适用于需要高阶光滑逼近的场景，如科学计算
我们推导出一个同时适用于函数及其任意阶导数的近似误差上界。该结果针对多元多项式、指数函数和倒数函数等初等函数，通过使用高斯误差线性单元（GELU）激活的前馈神经网络获得。分析还报告了网络规模、权重大小及在无穷远处的行为。研究从构造性乘法逼近入手，证明了对于给定逼近器，在域尺寸不断增加的情况下，误差界仍保持有效。基于此，我们获得了除法和指数函数的逼近保证，确保所得逼近器的所有高阶导数全局有界。
原文摘要 · Abstract (English)
We derive an approximation error bound that holds simultaneously for a function and all its derivatives up to any prescribed order. The bounds apply to elementary functions, including multivariate polynomials, the exponential function, and the reciprocal function, and are obtained using feedforward neural networks with the Gaussian Error Linear Unit (GELU) activation. In addition, we report the network size, weight magnitudes, and behavior at infinity. Our analysis begins with a constructive approximation of multiplication, where we prove the simultaneous validity of error bounds over domains of increasing size for a given approximator. Leveraging this result, we obtain approximation guarantees for division and the exponential function, ensuring that all higher-order derivatives of the resulting approximators remain globally bounded.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。