揭示核岭回归学习曲线的内在机制，提升理论理解与预测精度。
A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression
- 从核函数谱衰减、特征函数特性等角度分析学习曲线
- 验证高斯等价性，解释经典假设下的分析有效性
- 提出新界，在过/欠参数化场景下均优于已有结果
本文在极少假设下对核岭回归（KRR）的学习曲线进行了全面研究。贡献包括：1）分析核函数的关键属性，如谱的特征值衰减速率、特征函数特性以及核函数的光滑性；2）证明高斯等价性（GEP）成立，即当白化特征被标准高斯向量替代时，KRR的泛化性能不变，从而解释了以往基于高斯设计假设分析的成功原因；3）推导出新的泛化误差界，在独立/非独立特征向量、不同特征值衰减速率等多种设定下，均显著优于现有边界，涵盖过参数化和欠参数化情形。
原文摘要 · Abstract (English)
This paper conducts a comprehensive study of the learning curves of kernel ridge regression (KRR) under minimal assumptions. Our contributions are three-fold: 1) we analyze the role of key properties of the kernel, such as its spectral eigen-decay, the characteristics of the eigenfunctions, and the smoothness of the kernel; 2) we demonstrate the validity of the Gaussian Equivalent Property (GEP), which states that the generalization performance of KRR remains the same when the whitened features are replaced by standard Gaussian vectors, thereby shedding light on the success of previous analyzes under the Gaussian Design Assumption; 3) we derive novel bounds that improve over existing bounds across a broad range of setting such as (in)dependent feature vectors and various combinations of eigen-decay rates in the over/underparameterized regimes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。