用小波理论证明多种激活函数的神经网络逼近能力
Provable wavelet-based neural approximation
- 基于小波框架构建理论分析工具，适用于多种激活函数
- 推导出保证函数逼近的充分条件，并给出误差估计
- 支持光滑与非光滑激活函数，提升网络设计灵活性
本文建立了一个基于小波的理论框架，用于分析神经网络在多种激活函数下的通用逼近能力。借助同类型空间上的小波框架理论，我们推导出确保神经网络逼近给定空间中任意函数的激活函数充分条件，并提供误差估计。这些条件涵盖多种光滑激活函数，包括具有振荡特性的函数。此外，通过考虑光滑与非光滑激活函数间的 $L^2$-距离，我们建立了适用于非光滑激活函数的广义逼近结果，误差由该距离显式控制。这为网络架构设计提供了更大的灵活性。
原文摘要 · Abstract (English)
In this paper, we develop a wavelet-based theoretical framework for analyzing the universal approximation capabilities of neural networks over a wide range of activation functions. Leveraging wavelet frame theory on the spaces of homogeneous type, we derive sufficient conditions on activation functions to ensure that the associated neural network approximates any functions in the given space, along with an error estimate. These sufficient conditions accommodate a variety of smooth activation functions, including those that exhibit oscillatory behavior. Furthermore, by considering the $L^2$-distance between smooth and non-smooth activation functions, we establish a generalized approximation result that is applicable to non-smooth activations, with the error explicitly controlled by this distance. This provides increased flexibility in the design of network architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。