证明了小型神经网络在噪声数据上过拟合是温和的，不引发灾难性性能下降。
Provable Tempered Overfitting of Minimal Nets and Typical Nets
- 用最小权重数或随机方式训练二值权值网络，实现对噪声数据的完美拟合。
- 理论上证明无论哪种方法，过拟合程度都受控，泛化能力不会崩溃。
- 首次适用于深度网络且不限制输入维度的过拟合温和性分析，适合理论研究者。
我们研究了使用二值权重的全连接深度神经网络（NN）在噪声训练集上进行完美分类时的过拟合行为。考虑两种插值方式：最小神经网络（具有最少参数数量）和随机插值神经网络。对于这两种学习规则，我们证明了过拟合是温和的。分析基于一个新的关于与部分函数一致的阈值电路大小的界。据我们所知，这是首个适用于深度神经网络且不依赖极高或极低输入维度的良性过拟合理论结果。
原文摘要 · Abstract (English)
We study the overfitting behavior of fully connected deep Neural Networks (NNs) with binary weights fitted to perfectly classify a noisy training set. We consider interpolation using both the smallest NN (having the minimal number of weights) and a random interpolating NN. For both learning rules, we prove overfitting is tempered. Our analysis rests on a new bound on the size of a threshold circuit consistent with a partial function. To the best of our knowledge, ours are the first theoretical results on benign or tempered overfitting that: (1) apply to deep NNs, and (2) do not require a very high or very low input dimension.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。