研究高斯核无正则化回归在不同带宽和维度下的过拟合行为，揭示其一致性缺陷与良性过拟合条件。
Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality
- 分析带宽或维度随样本量变化时，最小范数插值解的过拟合特性。
- 证明固定维下无正则化解始终不一致，且噪声大时劣于零预测器。
- 首次给出基于高斯核的亚多项式维度增长下的良性过拟合例子。
我们研究高斯核无正则化回归（即无正则项的核岭回归）的过拟合行为，当带宽或输入维度随样本量变化时。在固定维度下，我们证明即使调整带宽，该无正则解也永远不一致，且在足够大的噪声下总是劣于零预测器。对于维度随样本量增加的情形，我们给出了任意维度与样本量比例下的过拟合行为通用刻画。基于此，我们首次提供了使用高斯核在亚多项式维度增长条件下的良性过拟合实例。所有结果均基于高斯普适性假设，并依赖于基于核特征结构的风险预测（非严格推导）。
原文摘要 · Abstract (English)
We consider the overfitting behavior of minimum norm interpolating solutions of Gaussian kernel ridge regression (i.e. kernel ridgeless regression), when the bandwidth or input dimension varies with the sample size. For fixed dimensions, we show that even with varying or tuned bandwidth, the ridgeless solution is never consistent and, at least with large enough noise, always worse than the null predictor. For increasing dimension, we give a generic characterization of the overfitting behavior for any scaling of the dimension with sample size. We use this to provide the first example of benign overfitting using the Gaussian kernel with sub-polynomial scaling dimension. All our results are under the Gaussian universality ansatz and the (non-rigorous) risk predictions in terms of the kernel eigenstructure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。