改进弹性网络的松弛方法,提升高维数据预测精度与稳定性。
Renet: Principled and Efficient Relaxation for the Elastic Net via Dynamic Objective Selection
- 通过动态选择目标函数实现自适应松弛,保持符号一致性。
- 在20个数据集上均优于标准弹性网络,尤其在低信噪比下表现更优。
- 适合高维、强共线性场景,兼顾计算效率与模型鲁棒性。
我们提出Renet,一种针对弹性网络的原理性推广松弛方法。尽管ℓ₁正则化适用于高维变量选择,ℓ₂惩罚可提供稳定性和解的唯一性,但标准弹性网络仍存在收缩偏差,常导致预测精度下降。为此,我们引入松弛框架:现有方法依赖简单的线性插值,忽略非线性正则路径几何,可能违反KKT条件。Renet通过自适应松弛过程,动态在凸混合与子路径重拟合间切换,强制符号一致性。进一步发现松弛与“一个标准误差”规则存在独特协同效应:松弛作为稳健去偏机制,使用户可在保持1-SE规则简洁性的同时,避免传统预测保真度损失。理论框架包含超高维情形的自动稳定性保障,并在20个合成与真实数据集上全面验证,结果表明Renet始终优于标准弹性网络,且在高维、低信噪比、高共线性条件下优于自适应弹性网络。借助自适应求解器后端,其统计性能提升同时保持与前沿坐标下降方法相当的计算效率。
原文摘要 · Abstract (English)
We introduce Renet, a principled generalization of the Relaxed Lasso to the Elastic Net family of estimators. While, on the one hand, $\ell_1$-regularization is a standard tool for variable selection in high-dimensional regimes and, on the other hand, the $\ell_2$ penalty provides stability and solution uniqueness through strict convexity, the standard Elastic Net nevertheless suffers from shrinkage bias that frequently yields suboptimal prediction accuracy. We propose to address this limitation through a framework called \textit{relaxation}. Existing relaxation implementations rely on naive linear interpolations of penalized and unpenalized solutions, which ignore the non-linear geometry that characterizes the entire regularization path and risk violating the Karush-Kuhn-Tucker conditions. Renet addresses these limitations by enforcing sign consistency through an adaptive relaxation procedure that dynamically dispatches between convex blending and efficient sub-path refitting. Furthermore, we identify and formalize a unique synergy between relaxation and the ``One-Standard-Error'' rule: relaxation serves as a robust debiasing mechanism, allowing practitioners to leverage the parsimony of the 1-SE rule without the traditional loss in predictive fidelity. Our theoretical framework incorporates automated stability safeguards for ultra-high dimensional regimes and is supported by a comprehensive benchmarking suite across 20 synthetic and real-world datasets, demonstrating that Renet consistently outperforms the standard Elastic Net and provides a more robust alternative to the Adaptive Elastic Net in high-dimensional, low signal-to-noise ratio and high-multicollinearity regimes. By leveraging an adaptive solver backend, Renet delivers these statistical gains while offering a computational profile that remains competitive with state-of-the-art coordinate descent implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。