对比四种正则化方法，发现高共线性下Lasso易失效，ElasticNet更稳健。
Choosing the Right Regularizer for Applied ML: Simulation Benchmarks of Popular Scikit-learn Regularization Frameworks
- 在13.4万次模拟中测试四种正则框架性能
- 高共线性时Lasso召回率仅0.18，ElasticNet达0.93
- 提供基于特征属性的正则化选型决策指南
本研究回顾了正则化的演进历程，从上世纪60年代的逐步回归发展到近年的形式化误差控制、非独立特征的结构化惩罚、贝叶斯方法及l0正则化等技术。我们通过134,400次模拟，在八个生产级机器学习模型构成的七维空间上，实证评估了Ridge、Lasso、ElasticNet和Post-Lasso OLS四种经典框架的表现。结果表明，当样本量与特征数比值足够高（n/p >= 78）时，Ridge、Lasso与ElasticNet在预测精度上几乎无差异。然而，当存在高共线性（条件数kappa高）且信噪比低时，Lasso召回率骤降至0.18，而ElasticNet仍保持0.93。因此建议在高kappa且小样本场景下避免使用Lasso或Post-Lasso OLS。最后，本文给出一个以目标为导向的选择指南，帮助机器学习工程师根据可观测的特征空间属性，选择最优的scikit-learn支持的正则化框架。
原文摘要 · Abstract (English)
This study surveys the historical development of regularization, tracing its evolution from stepwise regression in the 1960s to recent advancements in formal error control, structured penalties for non-independent features, Bayesian methods, and l0-based regularization (among other techniques). We empirically evaluate the performance of four canonical frameworks -- Ridge, Lasso, ElasticNet, and Post-Lasso OLS -- across 134,400 simulations spanning a 7-dimensional manifold grounded in eight production-grade machine learning models. Our findings demonstrate that for prediction accuracy when the sample-to-feature ratio is sufficient (n/p >= 78), Ridge, Lasso, and ElasticNet are nearly interchangeable. However, we find that Lasso recall is highly fragile under multicollinearity; at high condition numbers (kappa) and low SNR, Lasso recall collapses to 0.18 while ElasticNet maintains 0.93. Consequently, we advise practitioners against using Lasso or Post-Lasso OLS at high kappa with small sample sizes. The analysis concludes with an objective-driven decision guide to assist machine learning engineers in selecting the optimal scikit-learn-supported framework based on observable feature space attributes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。