深度学习过拟合仍能泛化，但现有解释缺乏理论支撑。
Benign interpolation and Occam's razor
- 用个体模型的性质替代模型类的复杂度来解释泛化
- 现有理论无法证明这些性质与泛化能力的必然联系
- 看似遵循奥卡姆剃刀，实则未经证明的假设
当代深度学习方法在完美拟合训练数据时仍能良好泛化，这一现象称为良性插值。经典统计学习理论无法解释此现象，促使学界提出多种新解释。这些新观点普遍诉诸插值模型中的简化偏好，常被类比为奥卡姆剃刀。本文为哲学背景读者厘清争议：新解释将模型个体属性视为简化性，但缺乏与泛化能力的可证关联。古典理论通过定理建立模型类简单性与泛化之间的联系，从而支持方法论上的简约规范。而新范式仅以‘简化’之名赋予个体属性合法性，实则将未加论证的假设伪装成熟悉的科学原则。
原文摘要 · Abstract (English)
Contemporary deep learning methods generalize well even when they fit their training data perfectly, a phenomenon known as benign interpolation. This phenomenon cannot be accounted for by classical statistical learning theory and has prompted a range of attempted new explanations in the statistics and machine learning literature. A common feature of these new proposals is an appeal to a simplicity preference among interpolating models, often presented as a form of Occam's razor. We clarify this debate for a philosophical audience and argue that this new appeal to simplicity creates an explanatory gap. The classical theory offers theorems which connect the simplicity of model classes to good generalization, thus underwriting methodological simplicity norms. The new accounts instead appeal to properties of individual models, which they interpret as a kind of simplicity. Lacking a provable connection to generalization, it is the name "simplicity" that does the work a theorem used to do, making a substantive and unargued assumption look like the application of a familiar methodological principle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。