从流形拓扑角度揭示神经网络表达能力的上限
A Theoretical Study of Neural Network Expressive Power via Manifold Topology
- 结合流形几何与拓扑特性分析网络规模需求
- 给出ReLU网络参数量的理论上限
- 适用于研究模型容量与数据结构关系的研究者
真实世界数据普遍被认为位于或靠近低维流形。当在数据流形上部署神经网络时，所需网络规模（即神经元数量）严重依赖于潜在隐含流形的复杂程度。尽管在理解流形几何属性方面已取得显著进展，但拓扑同样是流形的基本特征，不可忽视。本研究从潜在数据流形的角度探讨神经网络的表达能力，综合考虑数据流形的拓扑与几何特性，推导出ReLU神经网络规模的上界。
原文摘要 · Abstract (English)
A prevalent assumption regarding real-world data is that it lies on or close to a low-dimensional manifold. When deploying a neural network on data manifolds, the required size, i.e., the number of neurons of the network, heavily depends on the intricacy of the underlying latent manifold. While significant advancements have been made in understanding the geometric attributes of manifolds, it's essential to recognize that topology, too, is a fundamental characteristic of manifolds. In this study, we investigate network expressive power in terms of the latent data manifold. Integrating both topological and geometric facets of the data manifold, we present a size upper bound of ReLU neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。