arXiv:2603.28591math.DScs.LG2026-03

窄ResNet逼近能力受限,关键点会消失成'隧道效应'

Universal Approximation Constraints of Narrow ResNets: The Tunnel Effect

  • 分析窄ResNet在无输入增强下的逼近约束
  • 发现关键点常移至无穷远,导致分类失效
  • 适合研究网络结构与任务匹配的学者

我们从理论和数值两方面分析了窄残差神经网络(ResNets)的通用逼近限制。对于没有输入空间增强的深层神经网络,核心约束在于无法表示输入-输出映射的关键点。我们证明这一缺陷具有全局影响,并显示其表现通常为关键点移向无穷远,这在分类任务中被称为“隧道效应”。尽管ResNet比标准多层感知机(MLPs)更具表达力,但其能力强烈依赖于跳跃连接与残差通道之间的信号比例。我们建立了残差主导(接近MLP)和跳跃主导(接近神经微分方程)两种情形下的定量逼近界,这些界限显式依赖于通道比例和统一的网络权重界。低维示例进一步详细分析了不同ResNet范式及其架构-目标不兼容性对逼近误差的影响。

原文摘要 · Abstract (English)

We analyze the universal approximation constraints of narrow Residual Neural Networks (ResNets) both theoretically and numerically. For deep neural networks without input space augmentation, a central constraint is the inability to represent critical points of the input-output map. We prove that this has global consequences for target function approximations and show that the manifestation of this defect is typically a shift of the critical point to infinity, which we call the ``tunnel effect'' in the context of classification tasks. While ResNets offer greater expressivity than standard multilayer perceptrons (MLPs), their capability strongly depends on the signal ratio between the skip and residual channels. We establish quantitative approximation bounds for both the residual-dominant (close to MLP) and skip-dominant (close to neural ODE) regimes. These estimates depend explicitly on the channel ratio and uniform network weight bounds. Low-dimensional examples further provide a detailed analysis of the different ResNet regimes and how architecture-target incompatibility influences the approximation error.

ResNet逼近理论神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。