证明深度ReLU网络可最优逼近索博列夫与贝索夫函数，关键在稀疏向量编码新方法。
On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks
- 提出变宽深ReLU网络编码稀疏向量的新思路
- 在嵌入条件下实现最优逼近率 (WL)^(-2s/d)
- 适用于理论研究者与深度学习数学基础探索者
本文研究了在L^p([0,1]^d)范数下，深度ReLU神经网络以宽度W和深度L对索博列夫空间\mathcal{W}^{s,q}([0,1]^d)和贝索夫空间\mathcal{B}^s_{q,r}([0,1]^d)中的函数进行逼近的效率问题。近期研究已得出当p=q=∞时，逼近率为\mathcal{O}((WL)^{-2s/d})（忽略对数因子），以及在固定宽度下，当满足索博列夫嵌入条件1/q -1/p < s/d时，逼近率为\mathcal{O}(L^{-2s/d})。本文通过证明该速率在嵌入条件下仍成立，进一步推广了这些结果。此速率已被证明为最优（忽略对数因子）。论文的核心工具是利用变宽与变深的深度ReLU网络对稀疏向量进行新型编码，该方法本身可能具有独立意义。
原文摘要 · Abstract (English)
This paper studies the problem of how efficiently functions in the Sobolev spaces $\mathcal{W}^{s,q}([0,1]^d)$ and Besov spaces $\mathcal{B}^s_{q,r}([0,1]^d)$ can be approximated by deep ReLU neural networks with width $W$ and depth $L$, when the error is measured in the $L^p([0,1]^d)$ norm. This problem has been studied by several recent works, which obtained the approximation rate $\mathcal{O}((WL)^{-2s/d})$ up to logarithmic factors when $p=q=\infty$, and the rate $\mathcal{O}(L^{-2s/d})$ for networks with fixed width when the Sobolev embedding condition $1/q -1/p<s/d$ holds. We generalize these results by showing that the rate $\mathcal{O}((WL)^{-2s/d})$ indeed holds under the Sobolev embedding condition. It is known that this rate is optimal up to logarithmic factors. The key tool in our proof is a novel encoding of sparse vectors by using deep ReLU neural networks with varied width and depth, which may be of independent interest.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。