确定深度窄网络的最小宽度，揭示其通用逼近能力的理论极限。
Minimum Width of Deep Narrow Networks for Universal Approximation
- 通过几何方法构造新反例，证明最小宽度下界为 dy+1（当 dx<dy≤2dx）
- 发现 ELU/SELU 网络最小宽度上限为 max(2dx+1, dy)，且在 dy=2dx 时达到
- 首次统一证明多种激活函数下宽度范围，适用于网络架构设计
本文研究全连接神经网络实现通用逼近所需的最小宽度下界与上界，这对网络设计与训练具有重要意义。我们证明：对于 ELU、SELU 激活函数，最小宽度 w_min ≤ max(2dx+1, dy) 成立，且当 dy=2dx 时该上界可达；对于 LeakyReLU、ELU、CELU、SELU、Softplus 激活函数，有 dx+1 ≤ w_min ≤ dx+d_y，这是通过证明 ReLU 可被这些函数逼近得出的。此外，当激活函数为单射或可被单射函数序列一致逼近（如 ReLU）时，我们基于 Poincaré-Miranda 定理提出新几何构造法，重新证明 w_min ≥ dy + 1_{dx<dy≤2dx}，结果更直观。
原文摘要 · Abstract (English)
Determining the minimum width of fully connected neural networks has become a fundamental problem in recent theoretical studies of deep neural networks. In this paper, we study the lower bounds and upper bounds of the minimum width required for fully connected neural networks in order to have universal approximation capability, which is important in network design and training. We show that $w_{min}\leq\max(2d_x+1, d_y)$ also holds true for networks with ELU, SELU activation functions, and the upper bound of this inequality is attained when $d_y=2d_x$, where $d_x$, $d_y$ denote the input and output dimensions, respectively. Besides, we show that $d_x+1\leq w_{min}\leq d_x+d_y$ for networks with LeakyReLU, ELU, CELU, SELU, Softplus activation functions, by proving that ReLU activation function can be approximated by these activation functions. In addition, in the case that the activation function is injective or can be uniformly approximated by a sequence of injective functions (e.g., ReLU), we present a new proof of the inequality $w_{min}\ge d_y+\mathbf{1}_{d_x<d_y\leq2d_x}$ by constructing a more intuitive example via a new geometric approach based on Poincaré-Miranda Theorem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。