揭示设计矩阵零模式如何引发过参数回归中的多峰下降现象
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
- 通过图论方法分析协方差矩阵退化与变量相关性的影响
- 发现当设计矩阵存在特定零模式时，预测风险会出现多个峰值
- 适用于研究过参数模型泛化行为的理论方向
过去十年中，过参数化线性回归受到广泛关注。然而，大多数现有工作假设协变量独立且协方差矩阵非退化。本文放宽这两个假设，在趋于零正则化极限下推导了预测风险的确定性等价表达式。我们证明，协方差矩阵的退化性与变量间的依赖关系可导致多重下降现象，并刻画了对应峰值可能出现的位置。证明基于一种新颖的方差分布图表示法，表明最大匹配及关联二分图的Dulmage-Mendelsohn分解可识别出方差奇异的配置结构。
原文摘要 · Abstract (English)
Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate. In this paper, we relax both assumptions and derive deterministic equivalents for the prediction risk in a vanishing-ridge regime. We show that degeneracy of the covariance matrices and dependence can lead to multiple descent, and characterize where the corresponding peaks can occur. Our proofs use a novel graph representation of the variance profile. We show that maximum matchings and the Dulmage--Mendelsohn decomposition of the associated bipartite graph identify the configurations at which the variance becomes singular.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。