揭示交叉验证调参的渐近风险，与斯坦因无偏风险估计等价。
From Cross-Validation to SURE: Asymptotic Risk of Tuned Regularized Estimators
- 通过渐近分析，证明交叉验证调参等价于斯坦因无偏风险估计。
- 在正态均值模型中，预测损失收敛到收缩估计器的平方误差风险。
- 结果适用于高维统计学习，为调参性能提供精细刻画。
我们推导了由n折交叉验证（CV）调参的正则化经验风险最小化（ERM）估计器的渐近风险函数。此类估计器的样本外预测损失在分布上收敛至正态均值模型中收缩估计器的平方误差损失（风险函数），该收缩估计器由斯坦因无偏风险估计（SURE）调参。此风险函数比学习理论中常见的最坏情形后悔上界更精细，可量化风险如何随真实参数变化。关键中间步骤包括：(i) n折CV一致收敛至SURE；(ii) 尽管SURE通常有多个局部极小值，但其全局最小值在一般情况下是良好分离的。良好分离性保证了CV对SURE的一致收敛可转化为调参选择的一致收敛。
原文摘要 · Abstract (English)
We derive the asymptotic risk function of regularized empirical risk minimization (ERM) estimators tuned by $n$-fold cross-validation (CV). The out-of-sample prediction loss of such estimators converges in distribution to the squared-error loss (risk function) of shrinkage estimators in the normal means model, tuned by Stein's unbiased risk estimate (SURE). This risk function provides a more fine-grained picture of predictive performance than uniform bounds on worst-case regret, which are common in learning theory: it quantifies how risk varies with the true parameter. As key intermediate steps, we show that (i) $n$-fold CV converges uniformly to SURE, and (ii) while SURE typically has multiple local minima, its global minimum is generically well separated. Well-separation ensures that uniform convergence of CV to SURE translates into convergence of the tuning parameter chosen by CV to that chosen by SURE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。