揭示稀疏回归中过拟合的相变规律，解析对齐与噪声强度的影响。
Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
- 基于尖峰协方差模型分析最小范数插值解的泛化误差
- 发现当特征对齐度高时，增强信号反而引发灾难性过拟合
- 适用于理解高维线性/非线性模型的过拟合机制，适合理论学习者
本文研究了在尖峰协方差数据模型下，线性回归中最小范数插值解的泛化误差。通过精确推导泛化误差表达式，论文根据尖峰强度、样本-维度比 $c=d/n$（尤其当 $c \to \infty$ 时）及目标-尖峰对齐度，系统分类了良性、缓和与灾难性过拟合三种情形。值得注意的是，在目标与尖峰完全对齐的设定下，增加尖峰强度反而会引发灾难性过拟合，而非预期的良性过拟合。此外，研究发现目标-尖峰对齐并非总是有益，存在特定且有时反直觉的条件使其带来负面影响。这些现象在非线性模型中亦得到实证支持。
原文摘要 · Abstract (English)
This paper analyzes the generalization error of minimum-norm interpolating solutions in linear regression using spiked covariance data models. The paper characterizes how varying spike strengths and target-spike alignments can affect risk, especially in overparameterized settings. The study presents an exact expression for the generalization error, leading to a comprehensive classification of benign, tempered, and catastrophic overfitting regimes based on spike strength, the aspect ratio $c=d/n$ (particularly as $c \to \infty$), and target alignment. Notably, in well-specified aligned problems, increasing spike strength can surprisingly induce catastrophic overfitting before achieving benign overfitting. The paper also reveals that target-spike alignment is not always advantageous, identifying specific, sometimes counterintuitive, conditions for its benefit or detriment. Alignment with the spike being detrimental is empirically demonstrated to persist in nonlinear models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。