用物理和生物模型解释机器学习过拟合机制
Control of Overfitting with Physics
- 借鉴物理中的埃林公式控制梯度下降的过拟合
- 发现广义低能极小值对应低过拟合风险
- 将GAN类比捕食者-猎物模型，解释其泛化优势
尽管机器学习应用广泛，但对其效率的理论解释仍较少。本文通过物理学和生物学的类比，解释机器学习中的过拟合控制（或泛化性能）。对于随机梯度朗之万动力学（SGLD），我们证明了动力学理论中的埃林公式可实现算法稳定性下的过拟合控制——风险函数中宽而低自由能的极小值对应于低过拟合。对于生成对抗网络（GAN），我们建立其与生物学中捕食者-猎物模型的类比，该类比有助于解释为何选择宽似然极大值，并降低过拟合。
原文摘要 · Abstract (English)
While there are many works on the applications of machine learning, not so many of them are trying to understand the theoretical justifications to explain their efficiency. In this work, overfitting control (or generalization property) in machine learning is explained using analogies from physics and biology. For stochastic gradient Langevin dynamics, we show that the Eyring formula of kinetic theory allows to control overfitting in the algorithmic stability approach - when wide minima of the risk function with low free energy correspond to low overfitting. For the generative adversarial network (GAN) model, we establish an analogy between GAN and the predator-prey model in biology. An application of this analogy allows us to explain the selection of wide likelihood maxima and overfitting reduction for GANs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。