揭示了稀疏正则化模型中自动相关性确定的实现条件。
Empirical Bayes Estimation for Lasso-Type Regularizers: Analysis of Automatic Relevance Determination
- 通过经验贝叶斯方法推导分组Lasso模型的参数估计器。
- 发现特定条件下估计器发散，从而触发自动相关性确定机制。
- 明确了岭、Lasso、分组Lasso等模型实现ARD的通用条件。
本文研究具有非共轭稀疏诱导正则化的线性回归模型，如Lasso和分组Lasso。尽管经验贝叶斯方法可用来估计正则化参数，但其估计器的性质尚不明确。特别是关于自动相关性确定（ARD）机制发生的具体条件仍不清楚。本文推导了参数有限的分组Lasso正则化线性回归模型的经验贝叶斯估计器，证明在特定条件下估计器会发散，从而引发ARD机制。同时，我们还证明经验贝叶斯方法可在一般正则化线性回归模型中产生ARD机制，并阐明了岭、Lasso和分组Lasso等模型实现该机制的条件。
原文摘要 · Abstract (English)
This paper focuses on linear regression models with non-conjugate sparsity-inducing regularizers such as lasso and group lasso. Although the empirical Bayes approach enables us to estimate the regularization parameter, little is known on the properties of the estimators. In particular, many aspects regarding the specific conditions under which the mechanism of automatic relevance determination (ARD) occurs remain unexplained. In this paper, we derive the empirical Bayes estimators for the group lasso regularized linear regression models with limited parameters. It is shown that the estimators diverge under a specific condition, giving rise to the ARD mechanism. We also prove that empirical Bayes methods can produce the ARD mechanism in general regularized linear regression models and clarify the conditions under which models such as ridge, lasso, and group lasso can do so.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。