用统计学习理论证明：正则化通过偏好简单模型提升可靠性。
Statistical learning theory and Occam's razor: Regularization
- 基于统计学习理论，论证正则化是权衡拟合与简洁性的合理方法。
- 只有优先考虑简单性，才能获得理论可靠性和观测即真实保证。
- 为机器学习中的奥卡姆剃刀提供方法论支持，适合理论研究者阅读。
奥卡姆剃刀原则主张在归纳推理中偏好简单性，这一理念在科学哲学和机器学习领域均受到广泛关注，但其合理性始终难以确立。本文在早期‘核心论证’基础上，从统计学习理论出发，为正则化策略提供了一种合理辩护：为了获得理论可靠性及‘所见即所得’的保障，必须在拟合度与模型复杂度之间进行权衡，优先选择更简单的模型。这一论证构成一种真正的方法论正当性，既非纯粹功利主义的偏好，也非关于真理本质简单的形而上假设。
原文摘要 · Abstract (English)
The principle of Occam's razor, which instructs us to prefer simplicity in inductive inference, has attracted much scrutiny both in the philosophy of science and in machine learning. In either field, however, a justification for the principle has been elusive. In this paper, building on an earlier "core argument," I spell out a justification from statistical learning theory for the procedure of regularization: for trading off fit for simplicity. The means-ends argument is that in order to profit from theoretical reliability and "what-you-see-is-what-you-get" guarantees, one must implement a certain preference for simplicity over fit. This is a genuine methodological justification, which neither collapses to a purely pragmatic principle that we prefer simplicity for its own sake, nor to an ontological assumption that the truth is simple.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。