神经网络非黑箱，可通过极值动态映射提升泛化能力
Unraveling the Black Box of Neural Networks: A Dynamic Extremum Mapper
- 将数据映射到模型函数极值点，揭示泛化机制
- 参数越多，极值点数量越多，理论证明正相关
- 新算法解线性方程组，避开梯度消失与过拟合
我们指出神经网络并非黑箱，其泛化能力源于将数据集动态映射至模型函数极值点的能力。进一步证明神经网络的极值点数量与其参数数量呈正相关。为此提出一种与反向传播显著不同的新算法，主要通过求解线性方程组获得参数值。该框架可简洁解释并处理梯度消失、过拟合等复杂问题。
原文摘要 · Abstract (English)
We point out that neural networks are not black boxes, and their generalization stems from the ability to dynamically map a dataset to the extrema of the model function. We further prove that the number of extrema in a neural network is positively correlated with the number of its parameters. We then propose a new algorithm that is significantly different from back-propagation algorithm, which mainly obtains the values of parameters by solving a system of linear equations. Some difficult situations, such as gradient vanishing and overfitting, can be simply explained and dealt with in this framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。