让惩罚系数自动学习,提升非线性模型变量选择精度。
Proximal Iteration for Nonlinear Adaptive Lasso
- 将惩罚系数当作可优化变量,联合求解参数与系数
- 在真实数据上实现比固定结构更优的稀疏性和准确性
- 适合需要复杂稀疏模式的非线性建模任务
在平滑损失函数中加入ℓ₁正则项可同时实现估计与变量选择,并可通过近端梯度法高效求解。但ℓ₁正则存在偏差问题:非零参数会被低估。自适应Lasso通过为每个参数设置独立惩罚系数来缓解此问题,但在复杂模型中如何设定这些系数尚不明确。本文提出将惩罚系数视为待学习的额外决策变量,以最大后验方式联合优化其与可微损失函数的参数。该方法不仅降低估计偏差,还能通过惩罚系数的先验实现任意稀疏结构。我们在合成数据和真实数据(非高斯回归)上对比了该方法与特定稀疏结构实现的效果,结果表明本方法在速度和准确率上均具竞争力。进一步在两个案例研究中应用:新冠疫情疫苗接种行为、国际难民流动建模,验证了该方法对复杂问题和精细稀疏结构的适用性。
原文摘要 · Abstract (English)
Augmenting a smooth cost function with an $\ell_1$ penalty allows analysts to efficiently conduct estimation and variable selection simultaneously in sophisticated models and can be efficiently implemented using proximal gradient methods. However, one drawback of the $\ell_1$ penalty is bias: nonzero parameters are underestimated in magnitude, motivating techniques such as the Adaptive Lasso which endow each parameter with its own penalty coefficient. But it's not clear how these parameter-specific penalties should be set in complex models. In this article, we study the approach of treating the penalty coefficients as additional decision variables to be learned in a \textit{Maximum a Posteriori} manner, developing a proximal gradient approach to joint optimization of these together with the parameters of any differentiable cost function. Beyond reducing bias in estimates, this procedure can also encourage arbitrary sparsity structure via a prior on the penalty coefficients. We compare our method to implementations of specific sparsity structures for non-Gaussian regression on synthetic and real datasets, finding our more general method to be competitive in terms of both speed and accuracy. We then consider nonlinear models for two case studies: COVID-19 vaccination behavior and international refugee movement, highlighting the applicability of this approach to complex problems and intricate sparsity structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。