用梯度设计早停机制，让模型停止更科学。
ScoreStop: Gradient-based early stopping using functional score tests
- 基于梯度的函数得分检验，判断当前模型是否已最优。
- 在合成与真实数据上表现不输传统损失监控方法。
- 适用于隐式损失和依赖数据的损失场景，如排序与生存分析。
梯度提升决策树需要停止规则以避免过拟合。标准方法通过监控验证集损失，在连续若干轮未改善时停止，但耐心参数缺乏可解释性，且验证损失可能噪声大或由用户指定的梯度隐式定义。本文提出ScoreStop，一种基于梯度的早停策略，将每轮停止决策视为对原假设（当前预测器是最优总体风险估计）的检验。利用在验证数据上计算的函数得分检验，其统计量在更新方向上尺度不变，并在原假设下具有已知渐近分布。由于使用梯度而非损失值，该方法同样适用于隐式损失（如LambdaRank）及数据依赖损失（如通过影响函数实现的Cox回归）。在合成实验与真实数据基准测试中，ScoreStop的表现与基于损失的方法相当。
原文摘要 · Abstract (English)
Gradient boosted decision trees require a stopping rule to avoid overfitting. The standard rule monitors a validation loss and stops if the loss fails to improve for a fixed patience period. However, the patience parameter has no interpretable scale and validation losses can be noisy or implicitly defined by a user-specified gradient. We propose ScoreStop, a gradient-based early-stopping rule that casts the stopping decision at each iteration as a test of the null hypothesis that the current predictor is the population risk minimizer. We use a functional score test, computed on validation data, with a statistic that is scale-invariant in the update direction, with a known asymptotic distribution under the null. Because our test uses gradients rather than loss values, the same construction applies to implicit losses such as LambdaRank, and data-dependent losses such as Cox regression via influence functions. In synthetic experiments and real-data benchmarks, we show that ScoreStop is competitive with loss-based methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。