Python工具包实现迭代学习的隐式正则化，通过早停控制过拟合。
EarlyStopping: Implicit Regularization for Iterative Learning Procedures in Python
- 基于数据内顺序规则，对多种迭代算法实施早停
- 可追踪理论量并验证隐式正则化效果
- 适合研究模型泛化与迭代过程的开发者
迭代学习在机器学习和现代统计中广泛存在。为防止噪声在后续迭代中传播导致期望损失膨胀，通常需要正则化。通过早期停止实现隐式正则化已受到广泛关注。本文介绍EarlyStopping包，它为截断SVD、Landweber（梯度下降）、共轭梯度下降、L2-提升和回归树等常见迭代估计方法提供了一系列（样本内）序列早停规则。该工具包的核心特性是允许指定真实数据生成过程，并跟踪相关理论量。我们详细阐述了该包的实现原理，综述了近期理论文献中的基础进展，并演示如何使用该包探索隐式正则化的核心特性，复现文献结果。
原文摘要 · Abstract (English)
Iterative learning procedures are ubiquitous in machine learning and modern statistics. Regularision is typically required to prevent inflating the expected loss of a procedure in later iterations via the propagation of noise inherent in the data. Significant emphasis has been placed on achieving this regularisation implicitly by stopping procedures early. The EarlyStopping-package provides a toolbox of (in-sample) sequential early stopping rules for several well-known iterative estimation procedures, such as truncated SVD, Landweber (gradient descent), conjugate gradient descent, L2-boosting and regression trees. One of the central features of the package is that the algorithms allow the specification of the true data-generating process and keep track of relevant theoretical quantities. In this paper, we detail the principles governing the implementation of the EarlyStopping-package and provide a survey of recent foundational advances in the theoretical literature. We demonstrate how to use the EarlyStopping-package to explore core features of implicit regularisation and replicate results from the literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。