在含噪数据下，过参数化线性回归出现双下降现象，性能优于稳健方法。
Double descent for least-squares interpolation on contaminated data: A simulation study
- 用最小二乘插值法研究含噪数据下的过参数化模型表现
- 过参数化后泛化误差先升后降，最终超越稳健估计器
- 适用于研究模型复杂度与鲁棒性关系的机器学习研究者
过参数化模型虽按经典统计理论应过拟合，却常表现出优异泛化能力。'双下降'现象——即在达到一定模型复杂度后泛化误差下降——为此类现象提供了新视角。稳健统计关注含异常值的污染数据，因理想分布假设不成立，传统估计器易被严重扭曲。本文在含污染数据的线性回归设定下，探究双下降是否依然存在。通过比较高度非稳健的最小二乘插值估计器与多种稳健替代方法，发现即使在污染数据下，大规模过参数化仍能引发双下降现象，且最小二乘插值器的泛化性能优于所有稳健替代方法。
原文摘要 · Abstract (English)
Overparametrized models can exhibit an excellent generalization performance, although they should be prone to overfitting according to classical statistical theory. The discovery of the "double descent", indicating that the generalization error decreases after a certain model complexity has been reached, opened a new line of research. Robust statistics considers statistical estimation on contaminated data, which, due to assumptions that do not hold on real data, let data points appear as outliers w.r.t. the assumed "ideal" distribution, potentially severely distorting any classical estimator. We address the question whether a double descent phenomenon can be observed in a linear regression setting with contaminated training data. We compare the performance of the highly non-robust least-squares interpolation estimator with several robust alternatives. It turns out that large overparametrization indeed allows for a double descent phenomenon, resulting in a very good generalization performance of the least-squares interpolator, surpassing that of the robust alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。