揭示最小范数插值中良性过拟合的谱稳定性机制
Spectral-transport stability and benign overfitting for minimum norm interpolation
- 用谱运输距离衡量协方差谱扰动,建立稳定性理论
- 有效秩等关键量在谱扰动下呈李普希茨稳定,常数由特征值间隙决定
- 适用于研究数据表示变形或最优传输中的泛化行为
良性过拟合描述了最小范数插值估计器在完全拟合含噪数据的情况下仍能良好泛化的现象。现有刻画依赖于总体协方差算子的精细谱函数,即其特征值尾部的有效秩。本文研究当协方差谱受到扰动时这些刻画的稳定性,以特征测度间的Wasserstein距离量化扰动,称为谱运输。我们证明特征值尾部求和、尾部二阶矩以及支配良性过拟合的两个有效秩在谱运输距离下具有李普希茨稳定性,且显式常数由特征值间隙决定。由此得到三个结果:在对齐谱扰动下最小范数插值器的风险转移定理、在趋于零的谱运输扰动下良性过拟合分类保持不变的稳定性定理,以及基于样本协方差谱通过算子范数集中实现良性性经验认证的结果。该框架将良性过拟合与调和分析构造(如扩散图、散射表示)中的数据表示形变,以及线性化最优传输中的Wasserstein几何联系起来。三个谱族的数值实验验证了理论:有效秩索引的排序可预测经验超额风险的排序,且观测到的风险变化在谱的Wasserstein距离上接近李普希茨速率。
原文摘要 · Abstract (English)
Benign overfitting describes the ability of minimum norm interpolating estimators to generalize despite fitting noisy data exactly. Existing characterizations depend on delicate spectral functionals of the population covariance operator, namely the effective ranks of its eigenvalue tail. We study the stability of these characterizations when the covariance spectrum is perturbed, and we quantify perturbations with the Wasserstein distance between spectral measures, a viewpoint we call spectral transport. We prove that eigenvalue tail sums, tail second moments, and the two effective ranks that govern benign overfitting are Lipschitz stable with respect to the spectral-transport distance, with explicit constants driven by an eigenvalue gap. As consequences we obtain three results: a risk transfer theorem for the minimum norm interpolator under aligned spectral perturbations, a stability theorem showing that the benign overfitting classification is preserved under vanishing spectral-transport perturbations, and an empirical certification result in which sample covariance spectra certify benignity through operator norm concentration. The framework connects benign overfitting to harmonic analysis constructions such as diffusion maps and scattering representations, where covariance spectra are perturbed by deformations of the data representation, and to linearized optimal transport, where Wasserstein geometry is the natural metric on perturbations. Numerical experiments with three spectral families confirm the theory: the ordering of the effective rank indices predicts the ordering of the empirical excess risks, and the observed risk change scales at a near Lipschitz rate in the Wasserstein distance between spectra.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。