提出新型鲁棒稀疏惩罚方法,有效应对重尾噪声和异常值干扰。
Exponential Lasso: robust sparse penalization under heavy-tailed noise and outliers with exponential-type loss
- 采用指数型损失函数,在小误差时近似二次,大误差时平滑衰减
- 理论证明在理想与污染条件下均保持快速收敛速率
- 适合高维数据中存在异常值或重尾噪声的建模场景
在高维统计中,Lasso是变量选择与参数估计的核心方法。然而其依赖平方损失函数,对异常值和重尾噪声敏感,可能导致不可靠的选择与偏差估计。为此,本文提出Exponential Lasso,将指数型损失函数融入Lasso框架。该损失函数在小误差时保持近二次特性,大误差时平滑衰减,有效抑制极端异常值影响。理论上,该方法在理想条件下达到经典Lasso的收敛速率,同时在重尾污染下保持鲁棒性。计算上通过MM算法高效优化,每轮求解加权Lasso子问题。数值实验表明,该方法在污染数据中显著优于经典Lasso,且在高斯噪声下仍保持优异性能。代码实现于R包heavylasso,开源可获取。
原文摘要 · Abstract (English)
In high-dimensional statistics, the Lasso is a cornerstone method for simultaneous variable selection and parameter estimation. However, its reliance on the squared loss function renders it highly sensitive to outliers and heavy-tailed noise, potentially leading to unreliable model selection and biased estimates. To address this limitation, we introduce the Exponential Lasso, a novel robust method that integrates an exponential-type loss function within the Lasso framework. This loss function is designed to achieve a smooth trade-off between statistical efficiency under Gaussian noise and robustness against data contamination. Unlike other methods that cap the influence of large residuals, the exponential loss smoothly redescends, effectively downweighting the impact of extreme outliers while preserving near-quadratic behavior for small errors. We establish theoretical guarantees showing that the Exponential Lasso achieves strong statistical convergence rates, matching the classical Lasso under ideal conditions while maintaining its robustness in the presence of heavy-tailed contamination. Computationally, the estimator is optimized efficiently via a Majorization-Minimization (MM) algorithm that iteratively solves a series of weighted Lasso subproblems. Numerical experiments demonstrate that the proposed method is highly competitive, outperforming the classical Lasso in contaminated settings and maintaining strong performance even under Gaussian noise. Our method is implemented in the \texttt{R} package \texttt{heavylasso} available on Github: https://github.com/tienmt/heavylasso
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。