arXiv:2601.01295cs.LGcs.NA2026-01被引 1

提出新型函数空间,揭示深度网络如何用更弱平滑性实现高效逼近。

Sobolev Approximation of Deep ReLU Networks in Log-Barron Space

  • 定义对数加权Barron空间,弱化函数光滑性要求
  • 证明深度ReLU网络可实现$O(n^{-1/2})$逼近误差且依赖显式深度
  • 揭示深度与低正则性需求的协同作用,适合高维建模研究者

通用逼近定理表明神经网络可逼近任意连续函数;然而参数量可能随环境维度指数增长,难以解释深层模型在高维数据上的实际成功。Barron空间理论给出了解决方案:若目标函数属于Barron空间,两层网络以$ n $个参数可实现$ L^2 $范数下$ O(n^{-1/2}) $的逼近误差。但经典Barron空间$ \\'mathscr{B}^{s+1} $仍需强于Sobolev空间$ H^s $的正则性,现有深度敏感结果常受限于$ sL \leq 1/2 $等假设。本文引入对数加权Barron空间$ \mathscr{B}^{\log} $,其假设严格弱于任意$ s>0 $的$ \mathscr{B}^s $。针对该空间,我们研究嵌入性质并基于Rademacher复杂度进行统计分析。进一步证明$ \mathscr{B}^{\log} $中的函数可被深度ReLU网络以明确深度依赖的方式逼近。随后定义族$ \mathscr{B}^{s,\log} $,建立$ H^1 $范数下的逼近界,并识别保持该速率的最大深度尺度。结果阐明深度如何降低高效表示所需的正则性,为超越经典Barron设定的深层架构性能提供更精确解释,适用于当前高维问题的稳定应用。

原文摘要 · Abstract (English)

Universal approximation theorems show that neural networks can approximate any continuous function; however, the number of parameters may grow exponentially with the ambient dimension, so these results do not fully explain the practical success of deep models on high-dimensional data. Barron space theory addresses this: if a target function belongs to a Barron space, a two-layer network with $n$ parameters achieves an $O(n^{-1/2})$ approximation error in $L^2$. Yet classical Barron spaces $\mathscr{B}^{s+1}$ still require stronger regularity than Sobolev spaces $H^s$, and existing depth-sensitive results often assume constraints such as $sL \le 1/2$. In this paper, we introduce a log-weighted Barron space $\mathscr{B}^{\log}$, which requires a strictly weaker assumption than $\mathscr{B}^s$ for any $s>0$. For this new function space, we first study embedding properties and carry out a statistical analysis via the Rademacher complexity. Then we prove that functions in $\mathscr{B}^{\log}$ can be approximated by deep ReLU networks with explicit depth dependence. We then define a family $\mathscr{B}^{s,\log}$, establish approximation bounds in the $H^1$ norm, and identify maximal depth scales under which these rates are preserved. Our results clarify how depth reduces regularity requirements for efficient representation, offering a more precise explanation for the performance of deep architectures beyond the classical Barron setting, and for their stable use in high-dimensional problems used today.

深度学习函数逼近Barron空间

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。