证明了仅用一个激活函数的ResNet和ODENet具有通用逼近能力。
Universal approximation property of ODENet and ResNet with a single activation function
- 使用单一激活函数与仿射映射组合构建向量场。
- 在有限区间内可统一逼近任意一般向量场的ODENet。
- 为深度网络的表达能力提供了理论支持，适合研究者参考。
我们研究了基于单个激活函数的ODENet和ResNet的通用逼近性质。ODENet是将初值映射到有限区间内常微分方程（ODE）终值的映射，被视为一类深度学习系统（如ResNet）的数学模型。本文考虑向量场由激活函数与仿射变换的单一复合构成，这是实际机器学习系统中常见的选择。结果表明，此类具有受限向量场的ODENet和ResNet，能够对任意一般向量场的ODENet实现一致逼近。
原文摘要 · Abstract (English)
We study a universal approximation property of ODENet and ResNet. The ODENet is a map from an initial value to the final value of an ODE system in a finite interval. It is considered a mathematical model of a ResNet-type deep learning system. We consider dynamical systems with vector fields given by a single composition of the activation function and an affine mapping, which is the most common choice of the ODENet or ResNet vector field in actual machine learning systems. We show that such an ODENet and ResNet with a restricted vector field can uniformly approximate ODENet with a general vector field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。