研究过参数二分类中岭回归的泛化行为，揭示了其良性过拟合条件。
Benign Overfitting and the Geometry of the Ridge Regression Solution in Binary Classification
- 基于异向聚类分布建模，分析岭回归在标签噪声下的分类误差
- 当聚类均值尺度大时，良性过拟合条件与回归任务一致
- 标签噪声改变解的几何结构但不改变整体行为，适合理论学习者
本文研究过参数二分类任务中岭回归的行为。假设样本来自具有相反均值的各向异性类条件聚类分布，并允许训练标签存在恒定水平的标签翻转噪声。在聚类分布协方差矩阵尾部有效秩较高的条件下，我们刻画了岭回归的分类误差。结果表明，岭回归的行为在不同聚类均值尺度下有显著差异，当尺度很大时，良性过拟合的条件与回归任务相同。此外，我们还分析了标签噪声对最小范数插值器（MNI）的影响：该场景下最优分类器是聚类均值向量的线性变换，在无噪声情况下MNI近似学习此变换；而引入标签噪声会显著改变解的几何结构，但保持相同的定性行为。
原文摘要 · Abstract (English)
In this work, we investigate the behavior of ridge regression in an overparameterized binary classification task. We assume examples are drawn from (anisotropic) class-conditional cluster distributions with opposing means and we allow for the training labels to have a constant level of label-flipping noise. We characterize the classification error achieved by ridge regression under the assumption that the covariance matrix of the cluster distribution has a high effective rank in the tail. We show that ridge regression has qualitatively different behavior depending on the scale of the cluster mean vector and its interaction with the covariance matrix of the cluster distributions. In regimes where the scale is very large, the conditions that allow for benign overfitting turn out to be the same as those for the regression task. We additionally provide insights into how the introduction of label noise affects the behavior of the minimum norm interpolator (MNI). The optimal classifier in this setting is a linear transformation of the cluster mean vector and in the noiseless setting the MNI approximately learns this transformation. On the other hand, the introduction of label noise can significantly change the geometry of the solution while preserving the same qualitative behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。