一种无需调参的贝叶斯广义线性模型方法,提升稀疏逻辑回归预测效果。
A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression
- 通过变分推断直接优化后验均值与先验参数,减少优化变量数量。
- 在多种稀疏逻辑回归任务中表现优于主流方法,提升预测精度。
- 适用于指数族分布,统一框架避免为每类分布设计新算法。
我们提出一种灵活的贝叶斯广义线性模型经验贝叶斯拟合方法。具体采用一种新型均场变分推断(VI)方法,并在VI算法内估计先验分布,实现无调参。与传统VI优化后验密度不同,本方法直接优化后验均值和先验参数,降低优化变量数量,支持L-BFGS、随机梯度下降等可扩展算法。该方法自动依据先验与似然确定最优后验,不同于现有方法常假设高斯变分分布。所提框架统一适用于广泛指数族分布,无需为每对似然-先验组合开发专属VI方法。应用于稀疏逻辑回归,在大量数值实验中表现出更优预测性能,优于主流稀疏逻辑回归方法。
原文摘要 · Abstract (English)
We introduce a flexible empirical Bayes approach for fitting Bayesian generalized linear models. Specifically, we adopt a novel mean-field variational inference (VI) method and the prior is estimated within the VI algorithm, making the method tuning-free. Unlike traditional VI methods that optimize the posterior density function, our approach directly optimizes the posterior mean and prior parameters. This formulation reduces the number of parameters to optimize and enables the use of scalable algorithms such as L-BFGS and stochastic gradient descent. Furthermore, our method automatically determines the optimal posterior based on the prior and likelihood, distinguishing it from existing VI methods that often assume a Gaussian variational. Our approach represents a unified framework applicable to a wide range of exponential family distributions, removing the need to develop unique VI methods for each combination of likelihood and prior distributions. We apply the framework to solve sparse logistic regression and demonstrate the superior predictive performance of our method in extensive numerical studies, by comparing it to prevalent sparse logistic regression approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。