将神经网络视为可计算的流形，揭示其深层学习机制
Deep Manifold Part 1: Anatomy of Neural Network Manifold
- 用数值流形方法构建神经网络数学框架
- 发现网络具无限自由度和深度指数级学习能力
- 提出训练完成标准等三个基础问题，适合理论研究者
基于数值流形方法原理，我们建立了神经网络流形的数学框架——深度流形（Deep Manifold），并发现神经网络本质上是前向与反向计算的结合体，具有近似无限自由度、随深度呈指数增长的学习能力、自进化的边界条件以及训练中的隐含瓶颈。我们定义了神经网络学习空间与深度流形空间，并引入神经网络内在路径与固定点两个概念。同时提出三个根本性问题：1）如何定义训练完成？2）深度学习的收敛点（神经网络固定点）在哪里？3）在反问题中负时间至关重要，给定数据中的标记时间戳有多重要？
原文摘要 · Abstract (English)
Based on the numerical manifold method principle, we developed a mathematical framework of a neural network manifold: Deep Manifold and discovered that neural networks: 1) is numerical computation combining forward and inverse; 2) have near infinite degrees of freedom; 3) exponential learning capacity with depth; 4) have self-progressing boundary conditions; 5) has training hidden bottleneck. We also define two concepts: neural network learning space and deep manifold space and introduce two concepts: neural network intrinsic pathway and fixed point. We raise three fundamental questions: 1). What is the training completion definition; 2). where is the deep learning convergence point (neural network fixed point); 3). How important is token timestamp in training data given negative time is critical in inverse problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。