提出通用框架,统一证明各类深度网络的逼近能力。
A General Method for Proving Networks Universal Approximation Property
- 定义可逼近的模块化基础单元(UAM)
- 任意由UAM组成的深层网络均具逼近能力
- 适合理论研究者与架构设计者参考
深度学习模型架构高度多样化。现有方法针对每种架构(如全连接网络、CNN、Transformer)需独立构建数学形式并单独证明其通用逼近性,存在两大缺陷:一是新架构需重写完整证明,二是各证明彼此孤立,缺乏统一分析基础。本文提出一种通用且模块化的通用逼近性证明框架:定义具备逼近能力的基本单元(称为通用逼近模块,UAM),证明只要网络由此类模块构成,则整体仍具通用逼近性。同时,逼近过程可被理解为模块间逐步精炼的过程。该视角统一了对不同架构的理论分析,支持分步理解网络表达能力的演化机制。
原文摘要 · Abstract (English)
Deep learning architectures are highly diverse. To prove their universal approximation properties, existing works typically rely on model-specific proofs. Generally, they construct a dedicated mathematical formulation for each architecture (e.g., fully connected networks, CNNs, or Transformers) and then prove their universal approximability. However, this approach suffers from two major limitations: first, every newly proposed architecture often requires a completely new proof from scratch; second, these proofs are largely isolated from one another, lacking a common analytical foundation. This not only incurs significant redundancy but also hinders unified theoretical understanding across different network families. To address these issues, this paper proposes a general and modular framework for proving universal approximation. We define a basic building block (comprising one or multiple layers) that possesses the universal approximation property as a Universal Approximation Module (UAM). Under this condition, we show that any deep network composed of such modules inherently retains the universal approximation property. Moreover, the overall approximation process can be interpreted as a progressive refinement across modules. This perspective not only unifies the analysis of diverse architectures but also enables a step-by-step understanding of how expressive power evolves through the network.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。